Compensating for drift in the down conversion of high definition sequences to lower resolution sequences

ABSTRACT

In a reduced memory MPEG decoder, drift caused by the application of ½ pixel resolution motion vectors during prediction filtering is reduced. Received P and B frame and field DCT coded blocks are down converted. Reference pixels are selected based upon a received motion vector. The selected reference pixels are prediction filtered in order to produce a prediction output. If the motion vector is a ½ pixel resolution motion vector, a drift compensation value is added to the prediction output. Then, the prediction output is added to the down converted P and B frame and field DCT coded blocks in order to form lower resolution reconstructed pixel blocks.

RELATED APPLICATIONS

This application claims subject matter similar to the subject matterdisclosed in U.S. patent application Ser. No. 09/106,367 filed Jun. 29,1998 and in U.S. patent application Ser. No. 09/460,645 filed Dec. 15,1999.

TECHNICAL FIELD OF THE INVENTION

The present invention relates to the application of drift compensationduring the down conversion and decoding of high resolution encoded videofor display by a lower resolution receiver.

BACKGROUND OF THE INVENTION

The international standard ISO/IEC 13818-2 entitled “Generic Coding ofMoving Pictures and Associated Audio Information: Video” and “Guide tothe Use of the ATSC Digital Television Standard” describe a system,known as MPEG-2, for encoding and decoding digital video data. Thestandard allows for the encoding of video over a wide range ofresolutions, including higher resolutions commonly known as HDTV.According to this standard, digital video data is encoded as a series ofcode words in a complicated manner that causes the average length of thecode words to be much smaller than would be the case if, for example,each pixel were coded as an 8 bit value. This type of encoding is alsoknown as data compression.

In the system described above, encoded pictures are made up of pixels.Each 8×8 array of pixels is known as a block, and a 2×2 array of these8×8 blocks is termed a macroblock. Compression is achieved using thewell known techniques of prediction (motion estimation in the encoder,motion compensation in the decoder), two dimensional discrete cosinetransformation (DCT) which is performed on 8×8 blocks of pixels,quantization of DCT coefficients, and Huffman and run/level coding.However, prediction is not used for every picture. Thus, while Ppictures are encoded with prediction from previous pictures, and while Bpictures are encoded using prediction from either a previous or asubsequent picture, I pictures are encoded without prediction.

An MPEG-2 encoder is shown in simplified form in FIG. 1. Datarepresenting macroblocks of pixels are fed to both a subtractor 12 and amotion estimator 14. In the case of P pictures and B pictures, themotion estimator 14 compares each new macroblock to be encoded with themacroblocks in a reference picture previously stored in a referencepicture memory 16. The motion estimator 14 finds the macroblock in thestored reference picture that most closely matches the new macroblock.

The motion estimator 14 reads this matching macro-block (known as apredicted macroblock) out of the reference picture memory 16 and sendsit to the subtractor 12 which subtracts it, on a pixel by pixel basis,from the new macro-block entering the MPEG-2 encoder 10. The output ofthe subtractor 12 is an error, or residual, that represents thedifference between the predicted macroblock and the new macroblock beingencoded. This residual is often very small. The residual is transformedfrom the spatial domain by a two dimensional DCT 18. The DCTcoefficients resulting from the two dimensional DCT 18 are thenquantized by a quantization block 20 in a process that reduces thenumber of bits needed to represent each coefficient. Usually, manycoefficients are effectively quantized to zero. The quantized DCTcoefficients are Huffman and run/level coded by a coder 22 which furtherreduces the average number of bits per coefficient.

The motion estimator 14 also calculates a motion vector (mv) whichrepresents the horizontal and vertical displacements of the predictedmacroblock in the reference picture from the position of the newmacroblock in the current picture being encoded. It should be noted thatmotion vectors may have ½ pixel resolution which is achieved by linearinterpolation between adjacent pixels. The data encoded by the coder 22are combined with the motion vector data from the motion estimator 14and with other information (such as an indication of whether the pictureis an I, P or B picture), and the combined data are transmitted to areceiver that includes an MPEG-2 decoder 30 (shown in FIG. 2 anddiscussed below).

For the case of P pictures, the quantized DCT coefficients from thequantization block 20 are also supplied to an internal decoder loop thatrepresents a portion of the operation of the MPEG-2 decoder 30. Withinthis internal loop, the residual from the quantization block 20 isinverse quantized by an inverse quantization block 24 and is inverse DCTtransformed by an inverse discrete cosine transform (IDCT) block 26. Thepredicted macroblock, that is read out of the reference picture memory16 and that is supplied to the subtractor 12, is also added back to theoutput of the IDCT block 26 on a pixel by pixel basis by an adder 28,and the result is stored back into the reference picture memory 16 inorder to serve as a macroblock of a reference picture for predictingsubsequent pictures. The object of this internal loop is to have thedata in the reference picture memory 16 of the MPEG-2 encoder 10 matchthe data in the reference picture memory of the MPEG-2 decoder 30. Bpictures are not stored as reference pictures.

In the case of I pictures, no motion estimation occurs and the negativeinput to the subtractor 12 is forced to zero. In this case, thequantized DCT coefficients provided by the two dimensional DCT 18represent transformed pixel values rather than residual values, as isthe case with P and B pictures. As in the case of P pictures, decoded Ipictures are stored as reference pictures.

The MPEG-2 decoder 30 illustrated in FIG. 2 is a simplified showing ofan MPEC-2 decoder. The decoding process implemented by the MPEG-2decoder 30 can be thought of as the reverse of the encoding processimplemented by the MPEG-2 encoder 10. Accordingly, the received encodeddata is Huffman and run/level decoded by a Huffman and run/level decoder32. Motion vectors and other information are parsed from the data streamflowing through the Huffman and run/level decoder 32. The motion vectorsare fed to a motion compensator 34. Quantized DCT coefficients at theoutput of the Huffman and run/level decoder 32 are fed to an inversequantization block 36 and then to an IDCT block 38 which transforms theinverse quantized DCT coefficients back into the spatial domain.

For P and B pictures, each motion vector is translated by the motioncompensator 34 to a memory address in order to read a particularmacroblock (predicted macroblock) out of a reference picture memory 42which contains previously stored reference pictures. An adder 44 addsthis predicted macroblock to the residual provided by the IDCT block 38in order to form reconstructed pixel data. For I pictures, there is noprediction, so that the prediction provided to the adder 44 is forced tozero. For I and P pictures, the output of the adder 44 is fed back tothe reference picture memory 42 to be stored as a reference picture forfuture predictions.

The MPEG encoder 10 can encode sequences of progressive or interlacedpictures. For sequences of interlaced pictures, pictures may be encodedas field pictures or as frame pictures. For field pictures, one picturecontains the odd lines of the raster, and the next picture contains theeven lines of the raster. All encoder and decoder processing is done onfields. Thus, the DCT transform is performed on 8×8 blocks that containall odd or all even numbered lines. These blocks are referred to asfield DCT coded blocks.

On the other hand, for frame pictures, each picture contains both oddand even numbered lines of the raster. Macroblocks of frame pictures areencoded as frames in the sense that an encoded macroblock contains bothodd and even lines. However, the DCT performed on the four blocks withineach macroblock of a frame picture may be done in two different ways.Each of the four DCT transform blocks in a macroblock may contain bothodd and even lines (frame DCT coded blocks), or alternatively two of thefour DCT blocks in a macroblock may contain only the odd lines of themacroblock and the other two blocks may contain only the even lines ofthe macroblock (field DCT coded blocks). See ISO/IEC 13818-2, section6.1.3, FIGS. 6-13 and 6-14. The coding decision as to which way toencode a picture may be made adaptively by the MPEG-2 encoder 10 basedupon which method results in better data compression.

Residual macroblocks in field pictures are field DCT coded and arepredicted from a reference field. Residual macroblocks in frame picturesthat are frame DCT coded are predicted from a reference frame. Residualmacroblocks in frame pictures that are field DCT coded have two blockspredicted from one reference field and two blocks predicted from eitherthe same or the other reference field.

For sequences of progressive pictures, all pictures are frame pictureswith frame DCT coding and frame prediction.

MPEG-2, as described above, includes the encoding and decoding of videoat high resolution (HDTV). In order to permit people to use theirexisting NTSC televisions so as to view HDTV transmitted programs, it isdesirable to provide a decoder that decodes high resolution MPEG-2encoded data as reduced resolution video data for display on existingNTSC televisions. (Reducing the resolution of television signals isoften called down conversion decoding.) Accordingly, such a downconverting decoder would allow the viewing of HDTV signals withoutrequiring viewers to buy expensive HDTV displays.

There are known techniques for making a down converting decoder suchthat it requires less circuitry and is, therefore, cheaper than adecoder that outputs full HDTV resolution. One of these methods isdisclosed in U.S. Pat. No. 5,262,854. The down conversion techniquedisclosed there is explained herein in connection with a down convertor50 shown in FIG. 3. The down convertor 50 includes a Huffman andrun/level decoder 52 and an inverse quantization block 54 which operateas previously described in connection with the Huffman and run/leveldecoder 32 and the inverse quantization block 36 of FIG. 2. However,instead of utilizing the 8×8 IDCT block 38 as shown in FIG. 2, the downconvertor 50 employs a down sampler 56 which discards the forty-eighthigh order DCT coefficients of an 8×8 block and performs a 4×4 IDCT onthe remaining 4×4 array of DCT coefficients. This process is usuallyreferred to as DCT domain down sampling. The result of this downsampling is effectively a filtered and down sampled 4×4 block ofresidual samples (for P or B pictures) or pixels for I pictures.

For residual samples, a prediction is added by an adder 58 to theresidual samples from the down sampler 56 in order to produce a decodedreduced resolution 4×4 block of pixels. This block is saved in areference picture memory 60 for subsequent predictions. Accordingly,predictions will be made from a reduced resolution reference, whilepredictions made in the decoder loop within the encoder are made fromfull resolution reference pictures. This difference means that theprediction derived from the reduced resolution reference will differ bysome amount from the corresponding prediction made by the encoder,resulting in error in the residual-plus-prediction sum provided by theadder 58 (this error is referred to herein as prediction error).Prediction error may increase as predictions are made upon predictionsuntil the reference is refreshed by the next I picture.

A motion compensator 62 attempts to reduce this prediction error byusing the full resolution motion vectors, even though the referencepicture is at lower resolution. First, a portion of the referencepicture that includes the predicted macroblock is read from thereference picture memory 60. This portion is selected based on all bitsof the motion vector except the least significant bit. This predictedmacroblock is interpolated back to full resolution by a 2×2 predictionup sample filter 64. Using the full resolution motion vector (which mayinclude ½ pixel resolution), a predicted full resolution macroblock isextracted from the up sampled portion based upon all of the bits of themotion vector. Then, a down sampler 66 performs a 2×2 down sampling onthe extracted full resolution macro-block in order to match theresolution of the 4×4 IDCT output of the down sampler 56. In this way,the prediction from the reference picture memory 60 is up sampled tomatch the full resolution residual pixel structure allowing the use offull resolution motion vectors. Then, the full resolution referencepicture is down sampled prior to addition by the adder 58 in order tomatch the resolution of the down sampled residual from the down sampler56.

There are several known good prediction up sampling/down samplingmethods that tend to minimize the prediction error caused by up samplingreference pictures that have been down sampled with a 4×4 IDCT. Thesemethods typically involve use of a two dimensional filter having five toeight taps and tap values that vary both with the motion vector valuefor the predicted macroblock relative to the nearest macroblockboundaries in the reference picture, and with the position of thecurrent pixel being interpolated within the predicted macroblock. Such afilter not only up samples the reduced resolution reference to fullresolution and subsequently down samples in a single operation, but itcan also include ½ pixel interpolation (when required due to an oddvalued motion vector). (See, for example, “Minimal Error Drift inFrequency Scalability for Motion Compensated DCT Coding,” Mokry andAnastassiou, IEEE Transactions on Circuits and Systems for VideoTechnology, August 1994, and “Drift Minimization in Frequency ScaleableCoders Using Block Based Filtering,” Johnson and Princen, IEEE Workshopon Visual Signal Processing and Communication, Melbourne, Australia,September 1993.)

A more general derivation of minimum drift prediction filters by usingthe Moore-Penrose inverse of a block based down sampling filter isdescribed in “Minimum Drift Architectures for 3-Layer Scalable DTVDecoding,” Vetro, Sun, DaGraca and Poon, IEEE Transactions on ConsumerElectronics, August 1998.

The following example is representative of the prediction upsampling/down sampling filter described in the Mokry and Johnson papers.This example is a one dimensional example but is easily extended to twodimensions. Let it be assumed that pixels y1 and pixels y2 as shown inFIG. 4 represent two adjacent blocks in a down sampled referencepicture, and that the desired predicted block straddles the boundarybetween the two blocks. The pixels y1 are up sampled to the pixels p1 byusing a four tap filter with a different set of tap values for each ofthe eight calculated pixels p1. The pixels y2 are likewise up sampled tothe pixels p2 by using the same four tap filter. (If the motion vectorrequires ½ pixel interpolation, this interpolation is done using linearinterpolation to calculate in between pixel values based on the pixelsp1 and p2.) From these sixteen pixels p1 and pixels p2, an up sampledprediction consisting of eight pixels q can be read using the fullresolution motion vector. The pixels q are then filtered and downsampled to pixels q′ by an eight tap filter with a different set of tapvalues for each of the four pixels q′. The Johnson paper teaches how todetermine the optimum tap values for these filters given that thereference picture was down sampled by a four point IDCT. The tap valuesare optimum in the sense that the prediction error is minimized. TheJohnson and Mokry papers also show that the up sampling, linearinterpolation, and down sampling filters can be combined into a singleeight tap filter with tap values that depend on the motion vector valuerelative to the nearest macroblock boundaries in the reference picture,and that depend on the particular pixels q′ being calculated.Accordingly, this single eight tap filter allows four pixels q′ to becalculated directly from the eight pixels y1 and y2.

For methods of down sampling other than the four point IDCT, the Vetropaper describes how to determine the optimum tap values for the upsampling filter. This up sampling can also be combined with the linearinterpolation and down sampling operations to form a single predictionfilter.

The down convertor 50, while generally adequate for progressive pictureswith frame DCT coded blocks, does not address problems that arise whenattempting to down convert sequences of interlaced pictures with mixedframe and field DCT coded blocks. These problems arise with respect tovertical down sampling and vertical prediction filtering.

Let it be assumed that horizontal down sampling is performed in the DCTdomain using a four point horizontal IDCT. Vertical down sampling mayalso utilize a four point IDCT or some other method. For field pictures,the vertical down sampling operation is then performed on incoming fieldcoded blocks. For frame pictures, the vertical operation is performed ona mix of field and frame coded blocks. Thus, reference pictures may havebeen down sampled on a field basis, a frame basis, or a mix of both. Aspreviously explained, low drift prediction filtering may be derived fromthe down sampling filter. If different reference pictures are downsampled differently, they will require different matching predictionfilters.

Worse yet is the case of a reference picture containing a mix of fieldand frame down sampled blocks. A given required prediction may overlapboth types of blocks. This complication may be resolved by convertingall incoming blocks to either frames or fields before down sampling.This conversion will result in a consistent vertical structure forreference pictures so that the same prediction filter can always beused.

It has been suggested that all incoming pictures be converted to framesbefore performing vertical down sampling (see “Frequency Domain DownConversion of HDTV Using Adaptive Motion Compensation,” by Vetro, Sun,Bao and Poon, ICIP '97). Conversion to frames before performing verticaldown sampling will result in better vertical resolution than would fieldbased down sampling. However, frame based down sampling requiresadditional memory in the decoder because a first field must be storedwhen received in order to allow the second field to arrive so that frameblocks may be formed. Also, severe blocking artifacts in motionsequences may occur (see “Frequency Domain Down Conversion of HDTV Usingan Optimal Motion Compensation Scheme,” by Vetro and Sun, Journal ofImaging Science and Technology, August 1998).

An alternative, that would avoid these problems and that is suggested inthe latter paper, is to convert all incoming pictures to fields beforeperforming vertical down sampling. Therefore, field based processing isused herein such that incoming blocks which are frame coded are firstconverted to fields before vertical down sampling.

Also, it is well known that four point IDCT down sampling may causevisible artifacts. For progressive pictures, the degree of visibility isusually acceptable for both horizontal and vertical processing. Forinterlaced pictures using field based vertical down sampling, however,these artifacts may be much more visible. Thus, a technique other thanthe four point IDCT may be implemented for its field based vertical downsampling (a four point IDCT is still employed for horizontal downsampling).

Moreover, when a vertical or horizontal motion vector is odd valued, ½pixel interpolation is required when forming a prediction from a fullresolution reference or an up sampled reduced resolution reference. Adecoder that linearly combines ½ pixel interpolation with otherfiltering operations can exhibit a drift error towards a darker pictureuntil refreshed by the next I frame. The present invention is arrangedto compensate for this drift error.

SUMMARY OF THE INVENTION

In accordance with one aspect of the present invention, a method isprovided to reduce drift in connection with an application of ½ pixelresolution motion vectors during prediction filtering. The methodcomprises the following steps: a) down converting received P and B frameand field DCT coded blocks; b) selecting reference pixels based upon areceived motion vector; c) performing prediction filtering on theselected reference pixels to produce a prediction output; d) if themotion vector is a ½ pixel resolution motion vector, adding a driftcompensation value to the prediction output; and, e) adding theprediction output to the down converted P and B frame and field DCTcoded blocks in order to form lower resolution reconstructed pixelblocks.

In accordance with another aspect of the present invention, a method ofreconstructing pixel blocks from P and B DCT coded blocks comprises thefollowing steps: a) selecting reference pixels from a memory in responseto a motion vector; b) filtering the selected reference pixels toproduce a filtered output; c) if the motion vector has ½ pixelresolution, adding a drift compensation value to the filtered output;and, d) adding the filtered output to the P and B DCT coded blocks inorder to form the reconstructed pixel blocks.

In accordance with still another aspect of the present invention, amethod of reconstructing pixel blocks from P and B DCT coded blockscomprises the following steps: a) selecting reference pixels from amemory in response to a motion vector; b) if the motion vector has ½pixel resolution, adding drift compensation to the selected referencepixels; and, c) adding the selected reference pixels to the P and B DCTcoded blocks in order to form the reconstructed pixel blocks.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features and advantages of the present invention willbecome more apparent from a detailed consideration of the invention whentaken in conjunction with the drawings in which:

FIG. 1 is a simplified block diagram of a known MPEG-2 encoder;

FIG. 2 is a simplified block diagram of a known MPEG-2 decoder;

FIG. 3 is a block diagram of a known down conversion decoder for an HDTVapplication;

FIG. 4 shows an exemplary set of pixel data useful in describing thebackground of the present invention;

FIG. 5 is a block diagram of a down converting decoder according to thepresent invention;

FIG. 6 shows an embodiment of an IDCT module of FIG. 5;

FIGS. 7-9 show exemplary sets of pixel data useful in describing thepresent invention;

FIG. 10 shows a block diagram of a motion compensator of FIG. 5 inadditional detail; and,

FIGS. 11-12 show additional sets of pixel data useful in describing thepresent invention.

DETAILED DESCRIPTION I.—Down Conversion

A down converting decoder 100 according to an embodiment of the presentinvention is shown in FIG. 5. For vertical processing, the downconverting decoder 100 implements DCT domain frame to field conversion,a field based vertical eight point IDCT, vertical spatial interblockfiltering and down sampling, and complimentary vertical minimum driftprediction filtering. For horizontal processing, the down convertingdecoder 100 implements a horizontal four point IDCT for filtering anddown sampling, and complimentary horizontal minimum drift predictionfiltering.

A filtering and down sampling module 102 of the down converting decoder100 receives its input from the inverse quantizer 54 of FIG. 3. Thevertical processing performed by the filtering and down sampling module102 is described herein in detail. The vertical filtering consists of aneight tap low pass symmetric FIR filter which is a cubic filter derivedfrom equation (8) in “Reconstruction Filters in Computer Graphics,” byMitchell and Netravali, Siggraph 1988, with B=C=⅓. Accordingly, the tapvalues of this filter are given by the following equation:

[f] ₈ =[f(−3) . . . f(4)]=[−0.0074 −0.0117 0.1280 0.3911 0.3911 0.1280−0.0117 −0.0074]  (1)

This eight point vertical filter is effectively used to execute acontinuous interblock field based vertical filtering operation with thefilter [f]₈ operating on received residual and I field pixel blocks.

Vertical filtering and down sampling implemented by the filtering anddown sampling module 102 is given by the following equation and can beviewed as a vertical spatial domain operation on an entire field doneone column (of 540 pixels) at a time:

[d] _(270×540) [x] _(540×1) =[x′] _(270×1)  (2)

where [x] is a column of pixels from a field of pixels to be filteredand down sampled, [x′] is the resulting vertically filtered and downsampled column of pixels, and [d] is a down sampling operator given bythe following circulant matrix: $\begin{matrix}{\left( \lbrack d\rbrack \right)_{270 \times 540} = \begin{bmatrix}{{{f(0)}\quad \ldots \quad {f(4)}},{0\quad \ldots}} & {{\ldots \quad 0},{{f\left( {- 3} \right)}\quad \ldots \quad {f\left( {- 1} \right)}}} \\{{{f\left( {- 2} \right)}\quad \ldots \quad {f(4)}},{0\quad \ldots}} & {{\ldots \quad 0},{f\left( {- 3} \right)}} \\{0,{{f\left( {- 3} \right)}\quad \ldots \quad {f(4)}},{0\quad \ldots}} & {\ldots \quad 0} \\{0,0,0,{{f\left( {- 3} \right)}\quad \ldots \quad {f(4)}},{0\quad \ldots}} & {\ldots \quad 0} \\\quad & \quad \\{0\quad \ldots} & {{\ldots \quad 0},{{f\left( {- 3} \right)}\quad \ldots \quad {f(4)}},0,0,0} \\{0\quad \ldots} & {{\ldots \quad 0},{{f\left( {- 3} \right)}\quad \ldots \quad {f(4)}},0} \\{{f(4)},{0\quad \ldots}} & {{\ldots \quad 0},{{f\left( {- 3} \right)}\quad \ldots \quad {f(1)}}} \\{{{f(2)}\quad \ldots \quad {f(4)}},{0\quad \ldots}} & {{\ldots \quad 0},{{f\left( {- 3} \right)}\quad \ldots \quad {f(1)}}}\end{bmatrix}_{270 \times 540}} & (3)\end{matrix}$

(According to the nomenclature used here, capital letters are used todenote the DCT domain, and small letters are used to denote the spatialdomain.) Each row of [d] contains all zeroes except for the eightelements of the filter [f]₈. Also, the eight elements of the filter [f]₈in a row are shifted right by two places with respect to the previousrow.

In a decoder, it is not particularly practical for the operator[d]_(270×540) to operate on entire columns of entire fields. Therefore,in accordance with the present invention, and in a manner describedbelow, the filter [f]₈ operates vertically on residual and I blocks asthey arrive. This interblock vertical filtering and down sampling can becombined with intrablock vertical eight point IDCT in an efficientmanner.

IA1.—Field Pictures (All Macroblocks Field DCT Coded): VerticalProcessing

In order to vertically filter a particular eight point block column, theDCT coefficients for that block column, and for the block columns aboveand below it, must have been received and stored.

Let [X_(b)]_(8×1) be the eight point coefficient block column to bevertically filtered and down sampled in order to produce four filteredand down sampled pixels [X_(b)′]_(4×1). Then, let [X_(a)]_(8×1) and[X_(c)]_(8×1) be the block columns above and below [X_(b)].

Matrix [FT]_(8×8) is the eight point forward DCT transform matrix thatis used by the two dimensional DCT 18 of FIG. 1 in order to convertresidual and pixel values to DCT coefficients. Therefore, the rows of[FT]_(8×8) consist of the well known eight point DCT basis vectors. Thetranspose of [FT]_(8×8) gives the eight point IDCT transform matrix.Thus, [IT]_(8×8)=[FT]^(T). Let $\begin{matrix}{\left\lbrack {IT}^{3} \right) = \begin{bmatrix}\lbrack{IT}\rbrack & 0 & 0 \\0 & \lbrack{IT}\rbrack & 0 \\0 & 0 & \lbrack{IT}\rbrack\end{bmatrix}_{24 \times 24}} & (4)\end{matrix}$

The matrix [IT³ _(mid)]_(14×24) is defined as the middle fourteen rowsof [IT³] such that the top five rows and bottom five rows have beeneliminated from [IT³]. The matrix [IT³ _(mid]) _(14×24) is applied tothe eight point coefficient block column [X_(b)]_(8×1) as well as theblock columns [X_(a)]_(8×1) and [X_(c)]_(8×1) above and below [X_(b)]according to the following equation: $\begin{matrix}{{\left\lbrack {IT}_{mid}^{3} \right\rbrack \begin{bmatrix}X_{a} \\X_{b} \\X_{c}\end{bmatrix}}_{24 \times 1} = \lbrack x\rbrack_{14 \times 1}} & (5)\end{matrix}$

where [x]_(14×1) are the 14 full resolution pixels needed as inputs forthe down sampling filter in order to calculate the four down sampledpixels for block b (i.e., the block [x_(b)]_(4×1)). If [d]_(4×14) is adown sampling filter given by the following equation: $\begin{matrix}{\left( \lbrack d\rbrack \right)_{4 \times 14} = \begin{bmatrix}\left( \lbrack f\rbrack \right)_{8} & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & \left( \lbrack f\rbrack \right)_{8} & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 0 & \left( \lbrack f\rbrack \right)_{8} & 0 & 0 \\0 & 0 & 0 & 0 & 0 & 0 & \left( \lbrack f\rbrack \right)_{8}\end{bmatrix}_{4 \times 14}} & (6)\end{matrix}$

then applying this down sampling filter [d] to the fourteen pixels[x]_(14×1) yields the desired down sampled pixels [x_(b)′]_(4×1) (i.e. ,[d] [x]_(14×1)=[X_(b)′_(4×1)).

The IDCT and down sampling operations can be combined into a single stepper block column according to the following equation:

[d] _(4×14) [IT ³ _(mid)]_(14×24) =[Q _(dit)]_(4×24)  (7)

Applying this IDCT/down sampling filter [Q_(dit)]_(4×24) to thetwenty-four pixels of the three block columns [X_(a)], [X_(b)], and[X_(c)] yields the desired down sampled pixels [X_(b)′]_(4×1) accordingto the following equation: $\begin{matrix}{{\left( \left\lbrack Q_{dit} \right\rbrack \right)_{4 \times 24}\begin{bmatrix}X_{a} \\X_{b} \\X_{c}\end{bmatrix}}_{24 \times 1} = \left( \left\lbrack x_{b}^{\prime} \right\rbrack \right)_{4 \times 1}} & (8)\end{matrix}$

IA2.—Field Pictures: Two Dimensional Processing

The above discussion is easily extended to two dimensional 8×8 field DCTcoded blocks with vertical processing on each column using [Q_(dit)] asdescribed above and with horizontal processing on each row using a fourpoint horizontal IDCT. Accordingly, X_(b) is redefined to be an 8×8 DCTcoefficient block, and X_(a) and X_(c) are redefined to be the 8×8 DCTcoefficient blocks above and below the 8×8 DCT coefficient block X_(b),respectively.

As a first step in extending the down sampling filter of the presentinvention to two dimensional 8×8 field DCT coded blocks, the high orderhorizontal coefficients of each of the blocks X_(a), X_(b), and X_(c)are discarded so that each is an 8×4 array (i.e., the left half of eachblock is retained) so that [Z_(a)]_(8×4) is the left half of[X_(a)]_(8×8) and so that [Z_(b)]_(8×4) and [Z_(c)]_(8×4) are similarlydefined. These matrices are stacked and defined according to thefollowing equation: $\begin{matrix}{\left( \lbrack Z\rbrack \right)_{24 \times 4} = \begin{bmatrix}\left( \left\lbrack Z_{a} \right\rbrack \right)_{8 \times 4} \\\left( \left\lbrack Z_{b} \right\rbrack \right)_{8 \times 4} \\\left( \left\lbrack Z_{c} \right\rbrack \right)_{8{x4}}\end{bmatrix}} & (9)\end{matrix}$

Vertical filtering and down sampling of the pixels [Z]_(24×4) can thenbe performed in accordance with the following equation:

[G _(b)]_(4×4) =[Q _(dit)]_(4×24) [Z] _(24×4)  (10)

where [Q_(dit)]_(4×24) is the IDCT/down sampling filter given byequation (7). Horizontal IDCT domain filtering and down sampling maythen be applied to the results [G_(b)]_(4×4) of equation (10) accordingto the following equation: $\begin{matrix}{\left( \left\lbrack x_{b}^{\prime} \right\rbrack \right)_{4 \times 4} = {\left( \left\lbrack G_{b} \right\rbrack \right)_{4 \times 4}\left\lbrack {T4}\frac{\rbrack_{4 \times 4}}{\sqrt{2}} \right.}} & (11)\end{matrix}$

where [T4] is the four point DCT basis vector matrix containing the wellknown four point basis vectors.

IB1.—Frame Pictures: Vertical Processing: Assume that All Macroblocksare Frame DCT Coded

In the case of frame pictures in which all macroblocks are frame DCTcoded, it is preferable to implement frame to field conversion, fieldbased vertical eight point IDCT, field based vertical interblockfiltering, and down sampling in a single efficient step.

A macroblock column may be defined to be any sixteen point column (i.e.,two vertical block columns) of a 16×16 macroblock. In order to apply asingle interblock vertical operator, the DCT coefficients for themacroblock column, and for the macroblock columns above and below it,must have been received and stored. Accordingly, let [X_(b)]_(16×1) bethe sixteen point frame coded macroblock column (i.e., two eight pointvertical blocks) which is to be filtered and down sampled in order toproduce the eight field based pixels defined as [xtop_(b)′]_(4×1) forthe top field and [xbot_(b)′]_(4×1) for the bottom field. Also, let[X_(a)]_(16×1) and [X_(c)]_(16×1) be the frame DCT coded macroblockcolumns above and below [X_(b)]. The is frame DCT coded macroblockcolumns [X_(a)], [X_(b)], and [X_(c)] may be assembled in accordancewith the following equation: $\begin{matrix}{\left( \left\lbrack X_{frame} \right\rbrack \right)_{48 \times 1} = \begin{bmatrix}X_{a} \\X_{b} \\X_{c}\end{bmatrix}_{48 \times 1}} & (12)\end{matrix}$

An unshuffling operator that can pre-multiply a sixteen pixel frameordered column so as to reorder the even and odd pixel lines into topand bottom fields can be defined according to the following equation:$\begin{matrix}{\left( \lbrack{US}\rbrack \right)_{16 \times 16} = \begin{bmatrix}\begin{bmatrix}1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\\quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad \\0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0\end{bmatrix}_{8 \times 16} \\\begin{bmatrix}0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\\quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad \\0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1\end{bmatrix}_{8 \times 16}\end{bmatrix}_{16 \times 16}} & (13)\end{matrix}$

The unshuffling operator [US] may be applied to a frame orderedmacroblock column of pixels [x_(b)] which is the inverse DCT of [X_(b)]in accordance with the following equation: $\begin{matrix}{{\lbrack{US}\rbrack \left\lbrack x_{b} \right\rbrack}_{16 \times 1} = \begin{bmatrix}\left( \left\lbrack {xtop}_{b} \right\rbrack \right)_{8 \times 1} \\\left( \left\lbrack {xbot}_{b} \right\rbrack \right)_{8 \times 1}\end{bmatrix}_{16 \times 1}} & (14)\end{matrix}$

where [xtop_(b)] and [xbot_(b)] are the pixels [x_(b)] reordered intotop and bottom field pixels, respectively. If [FT8] is the eight pointDCT forward transform matrix with rows that are the known eight pointDCT basis vectors, the inverse of [FT8] is given by [FT8]^(T). Let$\begin{matrix}{\left\lbrack {IT8}_{2} \right\rbrack = \begin{bmatrix}{FT8}^{T} & \quad \\\quad & {FT8}^{T}\end{bmatrix}_{16 \times 16}} & (15)\end{matrix}$

where [IT8 ₂] is an IDCT operator for two vertically stacked eight pointDCT block columns. Then the operators [US] and [IT8 ₂] can be combinedinto a single operator [USIT] according to the following equation:

[USIT] _(16×16) =[US][IT8 ₂]  (16)

From equations (14) and (16), the following equation results:$\begin{matrix}{{\lbrack{USIT}\rbrack \left\lbrack X_{b} \right\rbrack}_{16 \times 1} = \begin{bmatrix}\left( \left\lbrack {xtop}_{b} \right\rbrack \right)_{8 \times 1} \\\left( \left\lbrack {xbot}_{b} \right\rbrack \right)_{8 \times 1}\end{bmatrix}_{16 \times 1}} & (17)\end{matrix}$

The top eight rows of [USIT] may be defined as [USIT_(top)]_(8×16), andthe bottom eight rows of [USIT] may be defined as [USIT_(botp)]_(8×16).These newly defined operators may be applied to [X_(b)] in accordancewith the following equations:

 [USIT _(top)]_(8×16) [X _(b)]_(16×1) =[xtop _(b)]_(8×1)  (18)

and

[USIT _(bot)]_(8×16) [X _(b)]_(16×1) =[xbot _(b)]_(8×1)  (19)

An operator [USIT_(top3)]_(24×48) is defined in accordance with thefollowing equation: $\begin{matrix}{\left( \left\lbrack {USIT}_{top3} \right\rbrack \right)_{24 \times 48} = \begin{bmatrix}\left\lbrack {USIT}_{top} \right\rbrack & 0 & 0 \\0 & \left\lbrack {USIT}_{top} \right\rbrack & 0 \\0 & 0 & \left\lbrack {USIT}_{top} \right\rbrack\end{bmatrix}_{24 \times 48}} & (20)\end{matrix}$

The operator [USIT_(top3)]_(24×48) may be applied to the coefficients inthe blocks X_(a), X_(b), and X_(c) according to the following equation:$\begin{matrix}{{\left\lbrack {USIT}_{top3} \right\rbrack \begin{bmatrix}X_{a} \\X_{b} \\X_{c}\end{bmatrix}}_{48 \times 1} = \begin{bmatrix}\left\lbrack {xtop}_{a} \right\rbrack \\\left\lbrack {xtop}_{b} \right\rbrack \\\left\lbrack {xtop}_{c} \right\rbrack\end{bmatrix}_{24 \times 1}} & (21)\end{matrix}$

However, only the middle fourteen pixels are required as inputs to thedown sampling filter (i.e., the top five and the bottom five pixels maybe discarded from the right hand side of equation (21)). Therefore, anoperator [MID_(top)]_(14×48) may be defined. The operator[MID_(top)]_(14×48) contains only the middle fourteen rows of[USIT_(top3)]. The mid top unshuffling operator [MID_(top)]_(14×48) maybe applied to the coefficients in the blocks X_(a), X_(b), and X_(c)according to the following equation: $\begin{matrix}{{\left( \left\lbrack {MID}_{top} \right\rbrack \right)_{14 \times 48}\begin{bmatrix}X_{a} \\X_{b} \\X_{c}\end{bmatrix}}_{48 \times 1} = \left( \lbrack{xtop}\rbrack \right)_{14 \times 1}} & (22)\end{matrix}$

where [xtop]_(14×1) are the unshuffled fourteen pixels needed as inputsto the top field down sampling filter in order to calculate the topfield down sampled and filtered pixels [Xtop_(b)′]_(4×1).

The mid bottom unshuffling operator [MID_(bot)]_(14×48) may be derivedin the same manner. This mid bottom unshuffling operator[MID_(bot)]_(14×48) may be applied to the coefficients in the blocksX_(a), X_(b), and X_(c) according to the following equation:$\begin{matrix}{{\left( \left\lbrack {MID}_{bot} \right\rbrack \right)_{14 \times 48}\begin{bmatrix}X_{a} \\X_{b} \\X_{c}\end{bmatrix}}_{48 \times 1} = \left( \lbrack{xbot}\rbrack \right)_{14 \times 1}} & (23)\end{matrix}$

where [xbot]_(14×1) are the unshuffled fourteen pixels needed as inputsto the bottom field down sampling filter in order to calculate thebottom field down sampled and filtered pixels [xbot_(b)′]_(4×1).

The down sampling filter [d]_(4×14) can be combined with theIDCT/unshuffling operators [MID] as IDCT, unshuffling, and down samplingfilters [QL_(top)]_(4×48) and [QL_(bot)]_(4×48) according to thefollowing equations;

[QL _(top)]_(4×48) =[d] _(4×14) [MID _(top)]_(14×48)  (24)

and

[QL _(bot)]_(4×48) =[d] _(4×14) [MID _(bot)]_(14×48)  (25)

The unshuffling, IDCT, and down sampling filters [QL_(top)]_(4×48) and[QL_(bot)]_(4×48) may be applied to the coefficients in the blocksX_(a), X_(b), and X_(c) according to the following equations in order toperform eight point vertical IDCT, frame to field conversion, verticalinterblock filtering, and down sampling in a single step:$\begin{matrix}{{\left\lbrack {QL}_{top} \right\rbrack_{4 \times 48}\begin{bmatrix}X_{a} \\X_{b} \\X_{c}\end{bmatrix}}_{48 \times 1} = {\left\lbrack {xtop}_{b}^{\prime} \right\rbrack_{4 \times 1}\quad {and}}} & (26) \\{{\left\lbrack {QL}_{bot} \right\rbrack_{4 \times 48}\begin{bmatrix}X_{a} \\X_{b} \\X_{c}\end{bmatrix}}_{48 \times 1} = \left\lbrack {xbot}_{b}^{\prime} \right\rbrack_{4 \times 1}} & (27)\end{matrix}$

The unshuffling and down sampling filters [QL_(top)]_(4×48) and[QL_(bot)]_(4×48) can be combined as a single unshuffling and downsampling filter according to the following equation: $\begin{matrix}{\begin{bmatrix}\left\lbrack {QL}_{top} \right\rbrack \\\left\lbrack {QL}_{bot} \right\rbrack\end{bmatrix}_{8 \times 48} = \left( \lbrack{QL}\rbrack \right)_{8 \times 48}} & (28)\end{matrix}$

Accordingly, the unshuffling and down sampling filter [QL]_(8×48) may beapplied to the coefficients in the blocks X_(a), X_(b), and X_(c) in asingle step according to the following equation: $\begin{matrix}{{\left( \lbrack{QL}\rbrack \right)_{8 \times 48}\begin{bmatrix}X_{a} \\X_{b} \\X_{c}\end{bmatrix}}_{48 \times 1} = \begin{bmatrix}\left\lbrack {xtop}_{b}^{\prime} \right\rbrack \\\left\lbrack {xbot}_{b}^{\prime} \right\rbrack\end{bmatrix}_{8 \times 1}} & (29)\end{matrix}$

IB2.—Frame Pictures: Vertical Processing; Assume that Some Macroblocksare Frame DCT Coded and Some are Field DCT Coded

As discussed above, frame pictures may contain a mix of frame and fieldDCT coded macroblocks. Thus, any of macroblock columns [X_(a)], X_(b)]or [X_(c)] may be frame or field coded. If it is assumed, for example,that [X_(a)] is field coded and that [X_(b)] and [X_(c)] are framecoded, then the field coded macroblock column [X_(a)] is receivedaccording to the following equation: $\begin{matrix}{\left( \left\lbrack X_{a} \right\rbrack \right)_{16 \times 1} = \begin{bmatrix}\left\lbrack {Xtop}_{a} \right\rbrack \\\left\lbrack {Xbot}_{a} \right\rbrack\end{bmatrix}} & (30)\end{matrix}$

Therefore, this macroblock column [X_(a)] need not be split into fieldcolumns as the macroblock columns [X_(b)] and [X_(c)] must be. As aresult, the unshuffling operator is not needed and the IDCT operator maybe applied according to the following equation: $\begin{matrix}{{\left( \left\lbrack {IT8}_{2} \right\rbrack \right)_{16 \times 16}\left\lbrack X_{a} \right\rbrack}_{16 \times 1} = \begin{bmatrix}\left( \left\lbrack {Xtop}_{a} \right\rbrack \right)_{8 \times 1} \\\left( \left\lbrack {Xbot}_{a} \right\rbrack \right)_{8 \times 1}\end{bmatrix}} & (31)\end{matrix}$

It should also be noted that [IT8_(2top)]_(8×16)[X_(a)]_(16×1)=[xtop_(a)]_(8×1) and that [IT8_(2bot)]_(8×16)[X_(a)]_(16×1)=[xbot_(a)]_(8×1).

A combined unshuffling and IDCT operator may be defined according to thefollowing equation (which is similar to equation (20): $\begin{matrix}{\left( \left\lbrack {USIT}_{top3} \right\rbrack \right)_{24 \times 48} = \begin{bmatrix}\left\lbrack {OP}_{topa} \right\rbrack & 0 & 0 \\0 & \left\lbrack {OP}_{topb} \right\rbrack & 0 \\0 & 0 & \left\lbrack {OP}_{topc} \right\rbrack\end{bmatrix}_{24 \times 48}} & (32)\end{matrix}$

where [OP_(topn)]=[USIT_(top)] if the vertical macroblock [X_(n)] isframe coded, and [OP_(topn)]=[IT8 _(2top)] if the vertical macroblock[X_(n)] is field coded. There are 2³=8 possible configurations for thisflexible operator [USIT_(top3)]. In the case of the field coded verticalmacroblock column [X_(a)] and frame coded vertical macroblock columns[X_(b)] and [X_(c)] of the above example, the operator [USI_(top3)] isgiven by the following equation: $\begin{matrix}{\left( \left\lbrack {USIT}_{top3} \right\rbrack \right)_{24 \times 48} = \begin{bmatrix}\left\lbrack {IT8}_{2{top}} \right\rbrack & 0 & 0 \\0 & \left\lbrack {USIT}_{top} \right\rbrack & 0 \\0 & 0 & \left\lbrack {USIT}_{top} \right\rbrack\end{bmatrix}_{24 \times 48}} & (33)\end{matrix}$

As before, the operator [MID_(top)]_(14×48) contains the middle fourteenrows of [USIT_(top3)]_(24×48). In the same manner, flexible operators[USIT_(bot3)]_(24×48) and [MID_(bot)]_(14×48) can be derived. The [MID]operators can be combined with the down sampling filter [d] in themanner of equations (24) and (25) so as to form flexible operators[QL_(top)] and [QL_(bot)] for any combination of frame or field DCTcoded macroblocks.

Thus, for any combination of frame or field DCT coded macroblocks, acombined operator which includes an eight point IDCT, frame to fieldconversion, and interblock filtering and down sampling can be devisedaccording to the following equations: $\begin{matrix}{{\left( \left\lbrack {QL}_{top} \right\rbrack \right)_{4 \times 48}\begin{bmatrix}X_{a} \\X_{b} \\X_{c}\end{bmatrix}}_{48 \times 1} = \left( \left\lbrack {xtop}_{b}^{\prime} \right\rbrack \right)_{4 \times 1}} & (34) \\{and} & \quad \\{{\left\lbrack {QL}_{bot} \right\rbrack_{4 \times 48}\begin{bmatrix}X_{a} \\X_{b} \\X_{c}\end{bmatrix}}_{48 \times 1} = \left\lbrack {xbot}_{b}^{\prime} \right\rbrack_{4 \times 1}} & (35)\end{matrix}$

The top and bottom [QL] operators can be combined in accordance with thefollowing equation: $\begin{matrix}{\begin{bmatrix}\left\lbrack {QL}_{top} \right\rbrack \\\left\lbrack {QL}_{bot} \right\rbrack\end{bmatrix}_{8 \times 48} = \left( \lbrack{QL}\rbrack \right)_{8 \times 48}} & (36)\end{matrix}$

Thus, the unshuffling and down sampling filter [QL]_(8×48) may beapplied to the coefficients in the blocks X_(a), X_(b), and X_(c) in asingle step per macroblock column according to the following equation:$\begin{matrix}{{\left( \lbrack{QL}\rbrack \right)_{8 \times 48}\begin{bmatrix}X_{a} \\X_{b} \\X_{c}\end{bmatrix}}_{48 \times 1} = \begin{bmatrix}\left\lbrack {xtop}_{b}^{\prime} \right\rbrack \\\left\lbrack {xbot}_{b}^{\prime} \right\rbrack\end{bmatrix}_{8 \times 1}} & (37)\end{matrix}$

IB3.—Frame Pictures: Two Dimensional Processing

The above discussion relates only to vertical processing of either framecoded or field coded frame pictures, but may be easily extended to twodimensional 16×16 macroblocks with vertical processing on each columnusing [QL]_(8×48) from equation (37) as described above and horizontalprocessing on each row of each 8×8 block with a four point IDCT.

Accordingly, let X_(a), X_(b), and X_(c) be redefined to be threevertically stacked 16×16 macroblocks according to the followingequations: $\begin{matrix}{\left( \left\lbrack X_{a} \right\rbrack \right)_{16 \times 16} = \begin{bmatrix}\left( \left\lbrack X_{a1} \right\rbrack \right)_{8 \times 8} & \left( \left\lbrack X_{a2} \right\rbrack \right)_{8 \times 8} \\\left( \left\lbrack X_{a3} \right\rbrack \right)_{8 \times 8} & \left( \left\lbrack X_{a4} \right\rbrack \right)_{8 \times 8}\end{bmatrix}} & (38) \\{and} & \quad \\{\left\lbrack X_{b} \right\rbrack_{16 \times 16} = \begin{bmatrix}\left\lbrack X_{b1} \right\rbrack_{8 \times 8} & \left\lbrack X_{b2} \right\rbrack_{8 \times 8} \\\left\lbrack X_{b3} \right\rbrack_{8 \times 8} & \left\lbrack X_{b4} \right\rbrack_{8 \times 8}\end{bmatrix}} & (39) \\{and} & \quad \\{\left\lbrack X_{c} \right\rbrack_{16 \times 16} = \begin{bmatrix}\left\lbrack X_{c1} \right\rbrack_{8 \times 8} & \left\lbrack X_{c2} \right\rbrack_{8 \times 8} \\\left\lbrack X_{c3} \right\rbrack_{8 \times 8} & \left\lbrack X_{c4} \right\rbrack_{8 \times 8}\end{bmatrix}} & (40)\end{matrix}$

The left and right halves of the macroblock [X_(a)]_(16×16) may beredefined in accordance with the following equations: $\begin{matrix}{\left( \left\lbrack X_{aL} \right\rbrack \right)_{16 \times 8} = \begin{bmatrix}\left( \left\lbrack X_{a1} \right\rbrack \right)_{8 \times 8} \\\left( \left\lbrack X_{a3} \right\rbrack \right)_{8 \times 8}\end{bmatrix}} & (41) \\{and} & \quad \\{\left\lbrack X_{aR} \right\rbrack_{16 \times 8} = \begin{bmatrix}\left\lbrack X_{a2} \right\rbrack_{8 \times 8} \\\left\lbrack X_{a4} \right\rbrack_{8 \times 8}\end{bmatrix}} & (42)\end{matrix}$

The macroblocks [X_(bL)]_(16×8), [X_(bR)]_(16×8), [X_(cL)]_(16×8), and[X_(cR)]_(16×8) may be similarly defined.

As a first step in developing an operator for performing verticalprocessing on each column using [QL]_(8×48) from equation (37) asdescribed above and horizontal processing on each row of each 8×8 blockwith a four point IDCT, the high order horizontal coefficients of each8×8 block [X] are discarded so that each 8×8 block [X] becomes a an 8×4block [Z]. Thus, [Z_(a1)]_(8×4) is defined as the left half of[X_(a1)]_(8×8), [Z_(a2)]_(8×4) is defined as the left half of[X_(a2)]_(8×8), [Z_(a3)]_(8×4) is defined as the left half of[X_(a3)]_(8×8), and [Z_(a4)]_(8×4) is defined as the left half of[X_(a4)]_(8×8). These 8×4 blocks can be combined in accordance with thefollowing equations: $\begin{matrix}{\left( \left\lbrack Z_{aL} \right\rbrack \right)_{16 \times 4} = \begin{bmatrix}\left( \left\lbrack Z_{a1} \right\rbrack \right)_{8 \times 4} \\\left( \left\lbrack Z_{a3} \right\rbrack \right)_{8 \times 4}\end{bmatrix}} & (43) \\{and} & \quad \\{\left\lbrack Z_{aR} \right\rbrack_{16 \times 4} = \begin{bmatrix}\left\lbrack Z_{a2} \right\rbrack_{8 \times 4} \\\left\lbrack Z_{a4} \right\rbrack_{8 \times 4}\end{bmatrix}} & (44)\end{matrix}$

[Z_(bL)]_(16×4), [Z_(bR)]_(16×4), [Z_(cL)]_(16×4), and [Z_(cR)]_(16×4)can be similarly defined. Then, field based vertical filtering and downsampling of the left and right halves of [X_(b)] is given by thefollowing equations (using the flexible operator [QL], from equation(32)): $\begin{matrix}{\left( \lbrack{QL}\rbrack \right)_{8 \times 48} = {\begin{bmatrix}\left\lbrack Z_{aL} \right\rbrack \\\left\lbrack Z_{bL} \right\rbrack \\\left\lbrack Z_{cL} \right\rbrack\end{bmatrix}_{48 \times 4} = {{\left( \lbrack{QL}\rbrack \right)_{8 \times 48}\left\lbrack Z_{L} \right\rbrack}_{48 \times 4} = \begin{bmatrix}\left( \left\lbrack {Gtop}_{bL} \right\rbrack \right)_{4 \times 4} \\\left( \left\lbrack {Gbot}_{bL} \right\rbrack \right)_{4 \times 4}\end{bmatrix}}}} & (45) \\{and} & \quad \\{\lbrack{QL}\rbrack_{8 \times 48} = {\begin{bmatrix}\left\lbrack Z_{aR} \right\rbrack \\\left\lbrack Z_{bR} \right\rbrack \\\left\lbrack Z_{cR} \right\rbrack\end{bmatrix}_{48 \times 4} = {{\left( \lbrack{QL}\rbrack \right)_{8 \times 48}\left\lbrack Z_{R} \right\rbrack}_{48 \times 4} = \begin{bmatrix}\left( \left\lbrack {Gtop}_{bR} \right\rbrack \right)_{4 \times 4} \\\left( \left\lbrack {Gbot}_{bR} \right\rbrack \right)_{4 \times 4}\end{bmatrix}}}} & (46)\end{matrix}$

Then, horizontal DCT domain filtering and down sampling is performedaccording to the following equations: $\begin{matrix}{\begin{bmatrix}\left\lbrack {xtop}_{bL}^{\prime} \right\rbrack_{4 \times 4} \\\left\lbrack {xbot}_{bL}^{\prime} \right\rbrack_{4 \times 4}\end{bmatrix} = {\begin{bmatrix}\left\lbrack {Gtop}_{bL} \right\rbrack_{4 \times 4} \\\left\lbrack {Gbot}_{bL} \right\rbrack_{4 \times 4}\end{bmatrix}\frac{\lbrack{T4}\rbrack_{4 \times 4}}{\sqrt{2}}}} & (47) \\{and} & \quad \\{\begin{bmatrix}\left\lbrack {xtop}_{bR}^{\prime} \right\rbrack_{4 \times 4} \\\left\lbrack {xbot}_{bR}^{\prime} \right\rbrack_{4 \times 4}\end{bmatrix} = {\begin{bmatrix}\left\lbrack {Gtop}_{bR} \right\rbrack_{4 \times 4} \\\left\lbrack {Gbot}_{bR} \right\rbrack_{4 \times 4}\end{bmatrix}\frac{\lbrack{T4}\rbrack_{4 \times 4}`}{\sqrt{2}}}} & (48)\end{matrix}$

where [T4] is the four point DCT basis vector matrix. Thus, the twodimensional down sampled and filtered macroblock is given by thefollowing expression: $\begin{matrix}\begin{bmatrix}\left\lbrack {xtop}_{bL}^{\prime} \right\rbrack_{4 \times 4} & \left\lbrack {xtop}_{bR}^{\prime} \right\rbrack_{4 \times 4} \\\left\lbrack {xbot}_{bL}^{\prime} \right\rbrack_{4 \times 4} & \left\lbrack {xbot}_{bR}^{\prime} \right\rbrack_{4 \times 4}\end{bmatrix} & (49)\end{matrix}$

IC.—IDCT Module: Block Diagram (FIG. 5, Module 102)

The filtering and down sampling module 102 of FIG. 5 is shown in moredetail in FIG. 6 and utilizes the above derived operators [Q_(dit)] forfield pictures and [QL] for frame pictures in order to perform verticalprocessing as well as the four point IDCT operator [T4] for horizontalprocessing. Thus, the filtering and down sampling module 102 can handleboth field pictures (containing all field DCT coded blocks) and framepictures containing only frame DCT coded macroblocks or a mix of fieldDCT coded macroblocks and frame DCT coded macroblocks. The output of thefiltering and down sampling module 102 is always field based filteredand down sampled. All blocks undergo the same horizontal processing (thefour point IDCT), but for vertical processing, the [QL] operator is usedwhen necessary to effectively convert the frame coding to field coding,and then to filter and down sample each field separately. It should benoted that matrix multiplication is not commutative (i.e., A*B does notin general equal B*A). Therefore, each matrix multiplier has a pre and apost input.

IC1.—Field Pictures

A coefficient discarder 120 of FIG. 6 drops the four high orderhorizontal coefficients from each row of every 8×8 DCT coefficientblock. A memory 122 stores three rows of I or residual coefficientmacroblocks. (A row of macroblocks consists of a rectangle of pixels onemacroblock high by the number of macroblocks needed to go from the leftto the right side of a picture.) In case of unsent residual macroblocks,all coefficients are assumed to be zero. Three vertically stacked blocks[Z]_(24×4) (see equation (9)) at a time are read from the memory 122 andare directed to a vertical matrix multiplier 124 by a router 126 inresponse to a DCT coding type (field or frame).

The vertical matrix multiplier 124 performs the eight point verticalIDCT with vertical interblock filtering and down sampling in accordancewith equation (10). Thus, the vertical matrix multiplier 124pre-multiplies the three vertically stacked blocks [Z]_(24×4) by[Q_(dit)]_(4×24) in order to produce [G_(b)]_(4×4). A selector 128directs the vertically processed block [G_(b)] to a horizontal matrixmultiplier 130. The horizontal matrix multiplier 130 performs ahorizontal four point IDCT on [G_(b)] in accordance with equation (11)so as to produce [x_(b)′].

IC2.—Frame Pictures

The coefficient discarder 120 drops the four high order horizontalcoefficients from each row of every 8×8 coefficient block as before. Thememory 122 stores three rows of I or residual coefficient macroblocks.In case of unsent residual macroblocks, all coefficients are assumed tobe zero. Three vertically stacked macroblocks consisting of[Z_(L)]_(48×4), then [Z_(R)]_(48×4) (see equations (45) and (46), forexample) at a time are read from the memory 122 and are directed by therouter 126 to a vertical matrix multiplier 132 in response to the DCTcoding type (field or frame). The router 126 also routs the DCT codingtype (field or frame) for each macroblock to QL module 134. Based on thecoding types of the three vertically stacked macroblocks comprising[Z_(L)] and [Z_(R)], the QL module 134 creates a flexible [QL] operatoras previously discussed for the “pre” input to the vertical matrixmultiplier 132.

The vertical matrix multiplier 132 performs an eight point verticalIDCT, frame to field conversion on frame coded macroblocks, andinterblock filtering on [Z_(L)], then on [Z_(R)] as in equations (45)and (46). The output from the vertical matrix multiplier 132 is first$\begin{bmatrix}\left\lbrack {Gtop}_{bL} \right\rbrack_{4 \times 4} \\\left\lbrack {Gbot}_{bL} \right\rbrack_{4 \times 4}\end{bmatrix}$ ${and}\quad {{{then}\begin{bmatrix}\left\lbrack {Gtop}_{bR} \right\rbrack_{4 \times 4} \\\left\lbrack {Gbot}_{bR} \right\rbrack_{4 \times 4}\end{bmatrix}}.}$

These output matrices are fed to a store and delay module 136 which thenprovides the following 4×4 blocks one at a time to horizontal matrixmultiplier 130 through the selector 128: [Gtop_(bL)], [Gbot_(bL)],[Gtop_(bR)], [Gbot_(bR)]. The horizontal matrix multiplier 130 performsa horizontal four point IDCT on the rows of each 4×4 input block toproduce the following corresponding 4×4 output blocks in accordance withequation (47) and (48): [xtop_(bL)′], [xbot_(bL)′], [xtop_(bR)′],[xbot_(bR)′]

II.—Prediction Up Sampling and Down Sampling

IIA.—Derivation of Vertical Up Sampling Filter Matrix [u]

The vertical prediction filtering implemented by the motion compensator104 of FIG. 5 is now explained. An up sampling filter [u] applied by themotion compensator 104 is determined from the well known Moore-Penroseinverse of the down sampling operator [d] from equations (2) and (3) inaccordance with the following equation:

[u] _(540×270) =[d] ^(T) {[d] ^(T) [d]} ⁻¹  (50)

The up sampling filter [u] is applied to a column of a reducedresolution reference picture [x′]_(270×1) from a memory 106 inaccordance with the following equation:

[u] _(540×270) [x′] _(270×1) =[{circumflex over (x)}] _(540×1)  (51)

where [{circumflex over (x)}]_(540×1) is the resulting up sampled columnof pixels. The structure of [u] is based on the following twenty-one tap(with thirteen non-zero taps) filters:

[g ₁(−10) . . . g ₁(10)]=[0 0 0 0 0 0 0 0 .0011 .0372 1.2542 −.4051.1546 −.0576 .0215 −.0081 .0030 −.0011 .0004 −.0002 .0001]

and

[g ₂(−10) . . . g ₂(10)]=[.0001 −.0002 .0004 −.0011 .0030 −.0081 .0215−.0576 .1546 −.4051 1.2542 .0372 .0011 0 0 0 0 0 0 0 0].

The filter [g₂] has the same tap values as the filter [g₁] except inreverse order, and g₁(0)=g₂(0)=1.2542. The up sampling filter [u] is acirculant matrix consisting of one step shifted versions of [g₁] and[g₂] on alternate rows. Accordingly, the up sampling filter [u] is givenby the following equation: $\begin{matrix}{\lbrack u\rbrack = \begin{bmatrix}{{{g_{1}(0)}\quad \ldots \quad {g_{1}(10)}},{0\quad \ldots}} & \quad & {{\ldots \quad 0},{{g_{1}\left( {- 10} \right)}\quad \ldots \quad {g_{1}\left( {- 1} \right)}}} \\{{{g_{2}(0)}\quad \ldots \quad {g_{2}(10)}},{0\quad \ldots}} & \quad & {{\ldots \quad 0},{{g_{2}\left( {- 10} \right)}\quad \ldots \quad {g_{2}\left( {- 1} \right)}}} \\{{{g_{1}\left( {- 1} \right)}\quad \ldots \quad {g_{1}(10)}},{0\quad \ldots}} & \quad & {{\ldots \quad 0},{{g_{1}\left( {- 10} \right)}\quad \ldots \quad {g_{1}\left( {- 2} \right)}}} \\{{{g_{2}\left( {- 1} \right)}\quad \ldots \quad {g_{2}(10)}},{0\quad \ldots}} & \quad & {{\ldots \quad 0},{{g_{2}\left( {- 10} \right)}\quad \ldots \quad {g_{2}\left( {- 2} \right)}}} \\\quad & \quad & \quad \\{0\quad \ldots} & {{g_{1}\left( {- 10} \right)}\quad {{\ldots g}_{1}(10)}} & {\ldots \quad 0} \\{0\quad \ldots} & {{g_{2}\left( {- 10} \right)}\quad \ldots \quad {g_{2}(10)}} & {\ldots \quad 0} \\\quad & \quad & \quad \\{{{g_{1}(2)}\quad \ldots \quad {g_{1}(10)}},{0\quad \ldots}} & \quad & {{\ldots \quad 0},{{g_{1}\left( {- 10} \right)}\quad \ldots \quad {g_{1}(1)}}} \\{{{g_{2}(2)}\quad \ldots \quad {g_{2}(10)}},{0\quad \ldots}} & \quad & {{\ldots \quad 0},{{g_{2}\left( {- 10} \right)}\quad \ldots \quad {g_{2}(1)}}} \\{{{g_{1}(1)}\quad \ldots \quad {g_{1}(10)}},{0\quad \ldots}} & \quad & {{\ldots \quad 0},{{g_{1}\left( {- 10} \right)}\quad \ldots \quad {g_{1}(0)}}} \\{{{g_{2}(1)}\quad \ldots \quad {g_{2}(10)}},{0\quad \ldots}} & \quad & {{\ldots \quad 0},{{g_{2}\left( {- 10} \right)}\quad \ldots \quad {g_{2}(0)}}}\end{bmatrix}_{540 \times 370}} & (52)\end{matrix}$

It may be possible to achieve a good enough prediction using only theeight (or fewer) largest value non-zero taps of g₁ and g₂. The fewernon-zero taps, the less reference data that must be read from the memory106 for each macroblock prediction. For example, adequate prediction maybe achieved with the filters given by the following equations:

[g ₁]₁₃=[0 0 0 0 0 .0372 1.2542 −.4051 .1546 −.0576 .0215 −.0081.0030]  (53)

and

[g ₂]₁₃=[.0030 −.0081 .0215 −.0576 .1546 −.4051 1.2542 .0372 0 0 0 00]  (54)

The vertical up sampling operation using the filters g₁ and g₂ ofequations (53) and (54) can be thought of as follows. To calculate theup sampled pixel above a particular reference pixel in a down sampledreference field [x′], the center value of the filter g₁ (i.e., the valueat g₁(0)) is aligned with that reference pixel and a sum of products isperformed between the non-zero taps and the aligned pixels. To calculatethe up sampled pixel below this particular reference pixel in the downsampled reference field [x′], the center value of the filter g₂ (i.e.,the value at g₂(0)) is aligned with that reference pixel and a sum ofproducts is performed between the non-zero taps and the aligned pixels.

In the down converting decoder 100, when an up sampled prediction isformed by the motion compensator 104, entire columns need not be readfrom the reference memory 106 to be filtered by [u]. Instead, onlyenough pixels need to be read to satisfy the combined need of aprediction filter formed from the up sampler filters [g₁] and [g₂] (each13 taps) and the down sampler [f] of equation (1) as described below.

IIB.—Combined Prediction Up Sampling and Down Sampling

IIB1.—Field Prediction for Field Pictures

As shown in FIG. 7, the first column represents a portion of a column ofpixels [x]_(50×1) from a field in the full resolution reference picture16 as it exists in the encoder 10 of FIG. 1. The second columnrepresents the corresponding portion of a column of pixels [x′]_(25×1)from that field in the down sampled reference 106 in the decoder loop ofFIG. 5. The vertical full resolution motion vector (vmv) for a desiredfield prediction points to the indicated pixel in the first column. Thismotion vector is in units of ½ pixel so that, if vmv is odd, it pointsbetween pixels at full resolution.

It is instructive to consider the modulo values of vmv as defined by:$\begin{matrix}{{vmvtype} = \left\lfloor \frac{{vmv}\quad \% \quad 4}{2} \right\rfloor} & (55)\end{matrix}$

where % means the modulo operation. The value of vmvtype will be 0 or 1,and FIG. 7 illustrates both cases. When vmvtype=0, the vertical motionvector vmv points to or just below the top pixel of a pair of pixels ina full resolution reference field (i.e., the pixels with the rectangularbox around them in the first column of FIG. 7). A pixel in the downsampled reference field (i.e., the pixel with the square box around itin the second column of FIG. 7) is between the pixels of this pair ofpixels. When vmvtype=1, the vertical motion vector vmv points to or justbelow the bottom pixel of this pair of pixels. Again, the pixel in thedown sampled reference field is between the pixels of this pair ofpixels. The low resolution motion vector lvmv, which is used to accessthe prediction from the down sampled reference field, is given by thefollowing equation: $\begin{matrix}{{lvmv} = \left\lfloor \frac{vmv}{4} \right\rfloor} & (56)\end{matrix}$

This low resolution motion vector lvmv points to the pixel in the downsampled reference field (i.e., the pixel with the square box around itin the second column of FIG. 7). The input to the prediction filter mustinclude that pixel in the down sampled reference field together with theeight pixels above it and the sixteen pixels below it. These twenty-fivepixels shown in the second column of FIG. 7 are the exact amount thatmust be read as the input to the vertical prediction filtering processin order to produce a predicted vertical field macroblock.

The next column in FIG. 7 is [{circumflex over (x)}]_(24×1) whichrepresents the up sampled field prediction produced by the filters g₁and g₂ which are arranged as shown in equation (52). The circled pixelsin the second column of FIG. 7 represent the upper and lower limitsbetween which the g₁ and g₂ filter center taps traverse for the upsampling convolution filtering operation. This up sampling filteringoperation is given by the following equation:

[u][x′] _(25×1) =[{circumflex over (x)}] _(24×1)  (57)

where [X′]_(25×1) represents a column of twenty-five pixels from thereference picture, [{circumflex over (x)}]_(24×1) represents the upsampled pixels, and the up sampling filter [u] is based upon equations(53) and (54) and is given by the following equation: $\begin{matrix}{\lbrack u\rbrack_{24 \times 25} = \begin{bmatrix}{\left\lbrack g_{2} \right\rbrack_{13},{0\quad \ldots}} & \quad & {\ldots \quad 0} \\{0,\left\lbrack g_{1} \right\rbrack_{13},{0\quad \ldots}} & \quad & {\ldots \quad 0} \\{0,\left\lbrack g_{2} \right\rbrack_{13},{0\quad \ldots}} & \quad & {\ldots \quad 0} \\\quad & \quad & \quad \\{0\quad \ldots} & \quad & {{\ldots \quad 0},\left\lbrack g_{1} \right\rbrack_{13},0} \\{0\quad \ldots} & \quad & {{\ldots \quad 0},\left\lbrack g_{2} \right\rbrack_{13},0} \\{0\quad \ldots} & \quad & {{\ldots \quad 0},\left\lbrack g_{1} \right\rbrack_{13}}\end{bmatrix}_{24 \times 25}} & (58)\end{matrix}$

After up sampling, a subsequent prediction down sampling operation isthen needed to produce the final desired down sampled field predictionmacroblock [{circumflex over (x)}′]_(8×1) column. This down sampling,which is described below, depends on vmvtype (as given by equation (56))and whether vmv is even or odd.

vmvtype=0

For the case of an even valued vertical motion vector vmv, the desireddown sampled field prediction is given by the convolution of [f]₈ with[{circumflex over (x)}]_(24×1) as follows:

[d ₀ ][{circumflex over (x)}] _(24×1) =[{circumflex over (x)}′]_(8×1)  (59)

where [{circumflex over (x)}]_(24×1) is a column of up sampled referencepixels to be down sampled, where [{circumflex over (x)}′]_(8×1) are theresulting down sampled reference pixels, where the prediction downsample filter [d₀] is given by the following equation: $\begin{matrix}{\left\lbrack d_{0} \right\rbrack_{8 \times 24} = \begin{bmatrix}{\lbrack f\rbrack_{8},{0\quad \ldots}} & \quad & {\ldots \quad 0} \\{0,0,\lbrack f\rbrack_{8},{0\quad \ldots}} & \quad & {\ldots \quad 0} \\\quad & \quad & \quad \\{0\quad \ldots} & \quad & {{\ldots \quad 0},\lbrack f\rbrack_{8},0,0}\end{bmatrix}_{8 \times 24}} & (60)\end{matrix}$

and where [f]₈ is from equation (1). The vertical prediction up sampleand down sample filtering can be combined into a single operation givenby the following equation:

[PL _(0−ev) ][x′] _(25×1) =[{circumflex over (x)}′] _(8×1)  (61)

where [x′]_(25×1) is a column of reference pixels from the memory 106,where [x′]_(8×1) are the pixels resulting from the prediction upsampling and down sampling, and where the up sampling and down samplingfilter [PL_(0−ev)] is given by the following equation:

[d ₀ ][u]=[PL _(0−ev)]_(8×25)  (62)

For the vmv odd valued case, up sampling must be followed by ½ pixellinear interpolation given by the following equation:

[LI][{circumflex over (x)}] _(24×1) =[{circumflex over (x)}_(i)]_(24×1)  (63)

where [{circumflex over (x)}]_(24×1) are the pixels resulting fromprediction up sampling, where [{circumflex over (x)}_(i)]_(24×1) are theprediction up sampled pixels resulting from linear interpolation, andwhere the linear interpolation filter [LI] is given by the followingequation: $\begin{matrix}{\lbrack{LI}\rbrack_{24 \times 24} = \begin{bmatrix}{1\text{/}2\quad 1\text{/}2\quad 0\quad \ldots} & \quad & {\ldots \quad 0} \\{0\quad 1\text{/}2\quad 1\text{/}2\quad 0\quad \ldots} & \quad & {\ldots \quad 0} \\\quad & \quad & \quad \\{0\quad \ldots} & \quad & {\ldots \quad 0\quad 1\text{/}2\quad 1\text{/}2} \\{0\quad \ldots} & \quad & {\ldots \quad 0}\end{bmatrix}_{24 \times 24}} & (64)\end{matrix}$

Then down sampling is performed in accordance with the followingequation:

[d ₀ ][{circumflex over (x)} _(i)]_(24×1) =[{circumflex over (x)}′]_(8×1)

Vertical prediction up sampling, linear interpolation, and predictiondown sampling can be combined into a single operation as given by thefollowing equation:

[PL _(0−odd) ][x′] _(25×1) =[{circumflex over (x)} _(i)′]_(8×1)  (65)

where the combined vertical prediction up sampling, linearinterpolation, and prediction down sampling filter [PL_(0−odd)]_(8×25)is given by the following equation:

[d ₀ ][LI][u]=[PL _(0−odd)]_(8×25)  (66)

vmvtype=1

For the case of an even valued vertical motion vector vmv, the desireddown sampled field prediction is given by the convolution of [f]₈ with[{circumflex over (x)}]_(24×1) as follows:

[d ₁ ][{circumflex over (x)}] _(24×1) =[{circumflex over (x)}′]_(8×1)  (67)

where the prediction down sampling filter [d₁] is given by the followingequation: $\begin{matrix}{\left\lbrack d_{1} \right\rbrack_{8 \times 24} = \begin{bmatrix}{0,\lbrack f\rbrack_{8},{0\quad \ldots}} & \quad & {\ldots \quad 0} \\{0,0,0,\lbrack f\rbrack_{8},{0\quad \ldots}} & \quad & {\ldots \quad 0} \\\quad & \quad & \quad \\{0\quad \ldots} & \quad & {{\ldots \quad 0},\lbrack f\rbrack_{8},0}\end{bmatrix}_{8 \times 24}} & (68)\end{matrix}$

The vertical prediction up sample and down sample filtering can becombined into a single operation according to the following equation:

[PL _(1−ev) ][x′] _(25×1) =[{circumflex over (x)}′] _(8×1)  (69)

where the combined vertical prediction up sample and down sample filter[PL_(1−ev)] is given by the following equation:

[d ₁ ][u]=[PL _(1−ev)]_(8×25)  (70)

For the case of an odd valued vertical motion vector vmv, up samplingmust be followed by ½ pixel linear interpolation given by equation (63).Then, down sampling is performed in accordance with equation (67). Thevertical up sampling, linear interpolation, and down sampling can becombined into a single operation given by the following equation:

[PL _(1−odd) ][x′] _(25×1) =[{circumflex over (x)} _(i)′]_(8×1)  (71)

where the combined vertical up sampling, linear interpolation, and downsampling filter [PL_(1−odd)] is given by the following equation:

[d ₁ ][LI][u]=[PL _(1−odd)]_(8×25)  (72)

IIB2.—Field Prediction for Frame Pictures

In the case of field prediction for frame pictures, two independentfield predictions for the frame macroblock are required. As shown inFIG. 8, the first column represents a portion of a column of pixels[x]_(42×1) from a field in the full resolution reference picture 16 asit exists in the encoder 10 of FIG. 1. The second column represents thecorresponding portion of a column of pixels [x′]_(21×1) from that fieldin the down sampled reference 106 in the decoder loop of FIG. 5. Thefull resolution vertical motion vector (vmv) for a desired fieldprediction points to the indicated pixel in the first column. Thismotion vector is in units of ½ pixel. Accordingly, if the fullresolution vertical motion vector vmv is odd valued, it points betweenpixels at full resolution.

It is again instructive to consider the modulo values of vmv as given byequation (55). As indicated, the value of vmvtype will be 0 or 1. FIG. 8illustrates both cases. When vmvtype=0, the vertical motion vector vmvpoints to or just below the top pixel of a pair of pixels in a fullresolution reference field (i.e., the pixels with the rectangular boxaround them in the first column of FIG. 8). A pixel in the down sampledreference field (i.e., the pixel with the square box around it in thesecond column of FIG. 8) is between the pixels of this pair of pixels.When vmvtype=1, the vertical motion vector vmv points to or just belowthe bottom pixel of this pair of pixels. Again, the pixel in the downsampled reference field is between the pixels of this pair of pixels.The low resolution motion vector lvmv, which is used to access theprediction from the down sampled reference field, is given by equation(56). This low resolution motion vector lvmv points to the pixel in thedown sampled reference field (second column) with the box around it. Theinput to the prediction filter must include that pixel, the eight pixelsabove it, and the twelve pixels below it. These twenty-one pixels arethe exact number of pixels that must be read as inputs to the verticalprediction filtering process in order to produce a predicted verticalfield block.

The third column in FIG. 8 is [{circumflex over (x)}]_(16×1) whichrepresents the up sampled field prediction produced by the filters g₁and g₂ given in equations (53) and (54). The circled pixels in thesecond column [x′]_(21×1) represent the upper and lower limits betweenwhich the center taps of the filters g₁ and g₂ traverse during the upsampling convolution operation.

The up sampling operation is given by the following equation:

[u][x′] _(21×1) =[{circumflex over (x)}] _(16×1)  (73)

where [x′]_(21×1) is twenty-one pixels from the reference picture storedin the memory 106, [{circumflex over (x)}]_(16×1) are the resulting upsampled pixels, and the up sampling filter [u] is given by the followingequation: $\begin{matrix}{\lbrack u\rbrack_{16 \times 21} = \begin{bmatrix}{\left\lbrack g_{2} \right\rbrack_{13},{0\quad \ldots}} & \quad & {\ldots \quad 0} \\{0,\left\lbrack g_{1} \right\rbrack_{13},{0\quad \ldots}} & \quad & {\ldots \quad 0} \\{0,\left\lbrack g_{2} \right\rbrack_{13},{0\quad \ldots}} & \quad & {\ldots \quad 0} \\\quad & \quad & \quad \\{0\quad \ldots} & \quad & {{\ldots \quad 0},\left\lbrack g_{1} \right\rbrack_{13},0} \\{0\quad \ldots} & \quad & {{\ldots \quad 0},\left\lbrack g_{2} \right\rbrack_{13},0} \\{0\quad \ldots} & \quad & {{\ldots \quad 0},\left\lbrack g_{1} \right\rbrack_{13}}\end{bmatrix}_{16 \times 21}} & (74)\end{matrix}$

A subsequent prediction down sampling operation is then needed toproduce the final desired down sampled field prediction [{circumflexover (x)}′]_(4×1). This down sampling, which is described below, dependson vmvtype and whether vmv is even or odd valued.

vmvtype=0

For the case where the vertical motion vector vmv is even valued, thedesired down sampled prediction is given by the convolution of [f]₈ with[{circumflex over (x)}]_(16×1) in accordance with the followingequation:

[d ₀ ][{circumflex over (x)}] _(16×1) =[{circumflex over (x)}′]_(4×1)  (75)

where [{circumflex over (x)}]_(16×1) are the up sampled pixels to bedown sampled, where [{circumflex over (x)}′]_(4×1) are the resultingdown sampled pixels, and where [d₀] is the down sampling filter given bythe following equation: $\begin{matrix}{\left\lbrack d_{0} \right\rbrack_{4 \times 16} = \begin{bmatrix}\lbrack f\rbrack_{8} & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & \lbrack f\rbrack_{8} & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 0 & \lbrack f\rbrack_{8} & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 0 & 0 & 0 & \lbrack f\rbrack_{8} & 0 & 0\end{bmatrix}_{4 \times 16}} & (76)\end{matrix}$

The vertical prediction up sample and down sample filtering can becombined into a single operation given by the following equation:

[PL _(0−ev) ][x′] _(21×1) =[{circumflex over (x)}′] _(4×1)  (77)

where the combined up sampling and down sampling filter [PL_(0−ev)] isgiven by the following equation:

[d ₀ ][u]=[PL _(0−ev)]_(4×21)  (78)

For the case where the vertical motion vector vmv is odd valued, upsampling must be followed by ½ pixel linear interpolation given by thefollowing equation:

[LI][{circumflex over (x)}] _(16×1) =[{circumflex over (x)}_(i)]_(16×1)  (79)

where [{circumflex over (x)}]_(16×1) are the up sampled pixels to belinearly interpolated, where [{circumflex over (x)}_(i)]_(16×1) are theresulting linearly interpolated pixels, and where the linearinterpolation filter is given by the following equation: $\begin{matrix}{\lbrack{LI}\rbrack_{16 \times 16} = \begin{bmatrix}{1\text{/}2\quad 1\text{/}2\quad 0\quad \ldots} & \quad & {\ldots \quad 0} \\{0\quad 1\text{/}2\quad 1\text{/}2\quad 0\quad \ldots} & \quad & {\ldots \quad 0} \\\quad & \quad & \quad \\{0\quad \ldots} & \quad & {\ldots \quad 0\quad 1\text{/}2\quad 1\text{/}2} \\{0\quad \ldots} & \quad & {\ldots \quad 0}\end{bmatrix}_{16 \times 16}} & (80)\end{matrix}$

Down sampling is then performed in accordance with the followingequation:

[d ₀ ][{circumflex over (x)} _(i)]_(16×1) =[{circumflex over (x)}′]_(4×1)  (81)

where [{circumflex over (x)}_(i)]_(16×1) are the up sampled and linearlyinterpolated pixels to be down sampled, where [{circumflex over(x)}′]_(4×1) are the resulting down sampled pixels, and where [d₀] isthe down sampling filter given by equation (76).

This vertical up sampling, linear interpolation, and down sampling canbe combined into a single operation given by the following equation:

[PL _(0−odd) ][x′] _(21×1) =[{circumflex over (x)} _(i)′]_(4×1)  (82)

where the combined vertical up sampling, linear interpolation, and downsampling filter [PL_(0−odd)] is given by the following equation:

[d ₀ ][LI][u]=[PL _(0−odd)]_(4×21)  (83)

vmvtype=1

For the case where the vertical motion vector vmv is even valued, thedesired down sampled field prediction is given by the convolution of[f]₈ with [{circumflex over (x)}]_(16×1) in accordance with thefollowing equation:

[d ₁ ][{circumflex over (x)}] _(16×1) =[{circumflex over (x)}′]_(4×1)  (84)

where the down sampling filter [d₁] is given by the following equation:$\begin{matrix}{\left\lbrack d_{1} \right\rbrack_{4 \times 16} = \begin{bmatrix}0 & \lbrack f\rbrack_{8} & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & \lbrack f\rbrack_{8} & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 0 & 0 & \lbrack f\rbrack_{8} & 0 & 0 & 0 \\0 & 0 & 0 & 0 & 0 & 0 & 0 & \lbrack f\rbrack_{8} & 0\end{bmatrix}_{4 \times 16}} & (85)\end{matrix}$

The vertical prediction up sample and down sample filtering can becombined into a single operation given by the following equation:

[PL _(1−ev) ][x′] _(21×1) =[{circumflex over (x)}] _(4×1)  (86)

where the combined up sampling and down sampling filter [PL_(1−ev)] isgiven by the following equation:

[d ₁ ][u]=[PL _(1−ev)]_(4×21)  (87)

For the case where the vertical motion vector vmv is odd valued, upsampling must be followed by ½ pixel linear interpolation given by thefollowing equation:

[LI] _(16×16) [{circumflex over (x)}] _(16×1) =[{circumflex over (x)}_(i)]_(16×1)  (88)

where the linear interpolation filter is given by equation (80). Then,down sampling can be performed in accordance with the followingequation:

[d ₁ ][{circumflex over (x)} _(i)]_(16×1) =[{circumflex over (x)}′]_(4×1)  (89)

Vertical up sampling, linear interpolation, and down sampling can becombined into a single operation given by the following equation:

[PL _(1−odd) ][x′] _(21×1) =[{circumflex over (x)} _(i)′]_(4×1)  (90)

where the combined vertical up sampling, linear interpolation, and downsampling filter [PL_(1−odd)] is given by the following equation:

[d ₁ ][LI][u]=[PL _(1−odd)]_(4×21)  (91)

IIB3.—Frame Prediction for Frame Pictures

For the case of frame prediction, two field predictions must be derivedto match the residual values which have been converted by the filteringand down sampling module 102 from a frame DCT coded macroblock to afield DCT coded macroblock. As shown in FIG. 9, the first columnrepresents a portion of a vertical column of pixels from a fullresolution reference frame 16 in the encoder 10 of FIG. 1. The secondcolumn represents the corresponding portion of a column of pixels[xtb′]_(42×1) from that frame in the down sampled reference from thememory 106 of the decoder loop. The full resolution motion vector (vmv)for a desired frame prediction points to the indicated pixel in thefirst column. This motion vector is in units of ½ pixel. If vmv is oddvalued, it points between pixels at full resolution.

The input to the prediction filter of the motion compensator 104 is thegroup of forty-two vertical pixels [xtb′] in the second column. Tocorrectly access the group of forty-two vertical pixels, the verticalcomponent of the full resolution motion vector is modified according tothe following equation: $\begin{matrix}{{lvmv} = {2\left\lfloor \frac{vmv}{8} \right\rfloor}} & (92)\end{matrix}$

This low resolution vertical motion vector lvmv will point to a topfield pixel (xtop′) in the down sampled reference frame. The input tothe prediction filter must include the sixteen pixels above it and thetwenty-five pixels below it.

The vertical motion vector vmv can be classified into one of four typesbased on a value vmvtype, each having a different spatial relationshipto the down sampled reference frame as shown in FIG. 9. Theseclassifications are given by the following equation; $\begin{matrix}{{vmvtype} = \left\lfloor \frac{{vmv}\quad \% \quad 8}{2} \right\rfloor} & (93)\end{matrix}$

where vmvtype can have values of 0, 1, 2, or 3 according to equation(93). From the definitions of [g₁]₁₃ and [g₂]₁₃ in equations (53) and(54), respectively, two twenty-five tap filters may be defined inaccordance with the following equations:

[gx ₁]₂₅ =[g ₁(0) 0 g ₁(1) 0 g ₁(2) 0 . . . g ₁(12)]  (94)

[gx ₂]₂₅ =[g ₂(0) 0 g ₂(1) 0 g ₂(2) 0 . . . g ₂(12)]  (95)

The up sampling filter [u] is then given by the following equation:$\begin{matrix}{\lbrack u\rbrack_{32 \times 42} = {\begin{bmatrix}\begin{bmatrix}{\left\lbrack {gx}_{2} \right\rbrack,0,0,{0\quad \ldots}} & \quad & {\ldots \quad 0} \\{0,0,\left\lbrack {gx}_{1} \right\rbrack,{0\quad \ldots}} & \quad & {\ldots \quad 0} \\{{0,0,\left\lbrack {gx}_{2} \right\rbrack,{0\quad \ldots}}\quad} & \quad & {\ldots \quad 0} \\\quad & \quad & \quad \\{0\quad \ldots} & \quad & {{\ldots \quad 0},\left\lbrack {gx}_{1} \right\rbrack,0,0,0} \\{{0\quad \ldots}\quad} & \quad & {{\ldots \quad 0},\left\lbrack {gx}_{2} \right\rbrack,0,0,0} \\{0\quad \ldots} & \quad & {{\ldots \quad 0},0,0,\left\lbrack {gx}_{1} \right\rbrack,0}\end{bmatrix}_{16 \times 42} \\\begin{bmatrix}{0,\left\lbrack {gx}_{2} \right\rbrack,0,0,{0\quad \ldots}} & \quad & {\ldots \quad 0} \\{0,0,0,\left\lbrack {gx}_{1} \right\rbrack,{0\quad \ldots}} & \quad & {\ldots \quad 0} \\{{0,0,0,\left\lbrack {gx}_{2} \right\rbrack,{0\quad \ldots}}\quad} & \quad & {\ldots \quad 0} \\\quad & \quad & \quad \\{0\quad \ldots} & \quad & {{\ldots \quad 0},\left\lbrack {gx}_{1} \right\rbrack,0,0} \\{{0\quad \ldots}\quad} & \quad & {{\ldots \quad 0},\left\lbrack {gx}_{2} \right\rbrack,0,0} \\{0\quad \ldots} & \quad & {{\ldots \quad 0},0,0,\left\lbrack {gx}_{1} \right\rbrack}\end{bmatrix}_{16 \times 42}\end{bmatrix}_{32 \times 42} = \begin{bmatrix}\left\lbrack u_{top} \right\rbrack \\\left\lbrack u_{bot} \right\rbrack\end{bmatrix}}} & (96)\end{matrix}$

Independently up sampling each field of [xtb′]_(42×1) from the downsampled reference frame in FIG. 9 is performed in accordance with thefollowing equation: $\begin{matrix}{{\lbrack u\rbrack_{32 \times 42}\left\lbrack {xtb}^{\prime} \right\rbrack}_{42 \times 1} = \begin{bmatrix}\left\lbrack {\hat{x}{top}} \right\rbrack_{16 \times 1} \\\left\lbrack {\hat{x}{bot}} \right\rbrack_{16 \times 1}\end{bmatrix}_{32 \times 1}} & (97)\end{matrix}$

Therefore, the up sampled prediction consists of [{circumflex over(x)}top]_(16×1) and [{circumflex over (x)}bot]_(16×1) which are the 3rdand 4th columns, respectively, in FIG. 9.

A subsequent prediction down sampling operation (on each up sampledfield) is then required in order to produce the final desired pair offield predictions. The down sampling, described below, depends onvmvtype and whether vmv is even or odd valued.

vmvtype=0, vmv even valued

The down sample filter for each field prediction is given by thefollowing equation: $\begin{matrix}{\left\lbrack d_{0 - {top}} \right\rbrack_{4 \times 16} = {\left\lbrack d_{0 - {bot}} \right\rbrack_{4 \times 16} = \begin{bmatrix}\lbrack f\rbrack_{8} & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & \lbrack f\rbrack_{8} & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 0 & \lbrack f\rbrack_{8} & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 0 & 0 & 0 & \lbrack f\rbrack_{8} & 0 & 0\end{bmatrix}_{4 \times 16}}} & (98)\end{matrix}$

The down sampling filter [d₀]_(8×32) may then be defined according tothe following equation: $\begin{matrix}{\left\lbrack d_{0} \right\rbrack_{8 \times 32} = \begin{bmatrix}\left\lbrack d_{0 - {top}} \right\rbrack_{4 \times 16} & 0 \\0 & \left\lbrack d_{0 - {bot}} \right\rbrack_{4 \times 16}\end{bmatrix}} & (99)\end{matrix}$

Independently down sampling each up sampled field results in the desiredfield predictions as given by the following equation: $\begin{matrix}{{\left\lbrack d_{0} \right\rbrack_{8 \times 32}\begin{bmatrix}\left\lbrack {\hat{x}{top}} \right\rbrack_{16 \times 1} \\\left\lbrack {\hat{x}{bot}} \right\rbrack_{16 \times 1}\end{bmatrix}}_{32 \times 1} = \begin{bmatrix}\left\lbrack {\hat{x}{top}^{\prime}} \right\rbrack_{4 \times 1} \\\left\lbrack {\hat{x}{bot}^{\prime}} \right\rbrack_{4 \times 1}\end{bmatrix}} & (100)\end{matrix}$

A combined up sample-down sample filter is given by the followingequation:

[PL _(0−ev)]_(8×42) =[d ₀]_(8×32) [u] _(32×42)  (101 )

This combined up sample-down sample filter may be used to produce thedesired field predictions as given by the following equation:$\begin{matrix}{{\left\lbrack {PL}_{0 - {ev}} \right\rbrack_{8 \times 42}\left\lbrack {xtb}^{\prime} \right\rbrack}_{42 \times 1} = \begin{bmatrix}\left\lbrack {\hat{x}{top}^{\prime}} \right\rbrack_{4 \times 1} \\\left\lbrack {\hat{x}{bot}^{\prime}} \right\rbrack_{4 \times 1}\end{bmatrix}} & (102)\end{matrix}$

vmvtype=0, vmv odd valued

As shown above, the independently up sampled field prediction is givenby the following expression: $\begin{matrix}\begin{bmatrix}\left\lbrack {\hat{x}{top}} \right\rbrack_{16 \times 1} \\\left\lbrack {\hat{x}{bot}} \right\rbrack_{16 \times 1}\end{bmatrix}_{32 \times 1} & (103)\end{matrix}$

The separately up sampled fields must be shuffled and linearlyinterpolated. The shuffle/interp operator is given by the followingequation: $\begin{matrix}{\lbrack{SLI}\rbrack_{32 \times 32} = \begin{bmatrix}\begin{bmatrix}{.5} & {0\quad \ldots} & \quad & \quad & {\ldots \quad 0} \\0 & {.5} & {0\quad \ldots} & \quad & {\ldots \quad 0} \\\quad & \quad & \quad & \quad & \quad \\{0\quad \ldots} & \quad & \quad & {\ldots \quad 0} & {.5}\end{bmatrix}_{16 \times 16} & \begin{bmatrix}{.5} & {0\quad \ldots} & \quad & \quad & {\ldots \quad 0} \\0 & {.5} & {0\quad \ldots} & \quad & {\ldots \quad 0} \\\quad & \quad & \quad & \quad & \quad \\{0\quad \ldots} & \quad & \quad & {\ldots \quad 0} & {.5}\end{bmatrix}_{16 \times 16} \\\begin{bmatrix}0 & {.5} & {0\quad \ldots} & \quad & {\ldots \quad 0} \\0 & 0 & {.5} & {0\quad \ldots} & {\ldots \quad 0} \\\quad & \quad & \quad & \quad & \quad \\{0\quad \ldots} & \quad & \quad & {\ldots \quad 0} & {.5} \\{0\quad \ldots} & \quad & \quad & \quad & {\ldots \quad 0}\end{bmatrix}_{16 \times 16} & \begin{bmatrix}{.5} & {0\quad \ldots} & \quad & \quad & {\ldots \quad 0} \\0 & {.5} & {0\quad \ldots} & \quad & {\ldots \quad 0} \\\quad & \quad & \quad & \quad & \quad \\{0\quad \ldots} & \quad & {\ldots \quad 0} & {.5} & 0 \\{0\quad \ldots} & \quad & \quad & \quad & {\ldots \quad 0}\end{bmatrix}_{16 \times 16}\end{bmatrix}_{32 \times 32}} & (104)\end{matrix}$

The shuffling and linear interpolation performed on the up sampledfields is given by the following equation: $\begin{matrix}{{\lbrack{SLI}\rbrack_{32 \times 32}\begin{bmatrix}\left\lbrack {\hat{x}{top}} \right\rbrack_{16 \times 1} \\\left\lbrack {\hat{x}{bot}} \right\rbrack_{16 \times 1}\end{bmatrix}}_{32 \times 1} = \begin{bmatrix}\left\lbrack {\hat{x}{topi}} \right\rbrack_{16 \times 1} \\\left\lbrack {\hat{x}{boti}} \right\rbrack_{16 \times 1}\end{bmatrix}_{32 \times 1}} & (106)\end{matrix}$

Independently down sampling each up sampled, shuffled, and linearlyinterpolated field results in the desired field predictions given by thefollowing equation: $\begin{matrix}{{\left\lbrack d_{0} \right\rbrack_{8 \times 32}\begin{bmatrix}\left\lbrack {\hat{x}{topi}} \right\rbrack_{16 \times 1} \\\left\lbrack {\hat{x}{boti}} \right\rbrack_{16 \times 1}\end{bmatrix}}_{32 \times 1} = \begin{bmatrix}\left\lbrack {\hat{x}{topi}^{\prime}} \right\rbrack_{4 \times 1} \\\left\lbrack {\hat{x}{boti}^{\prime}} \right\rbrack_{4 \times 1}\end{bmatrix}_{8 \times 1}} & (107)\end{matrix}$

where the down sampling filter [d₀]_(8×32) is given by equation (99).

The operations of the up sampling filter [u], the shuffling and linearlyinterpolating filter [SLI], and the down sampling filter [d₀] may becombined according to the following equation:

[PL _(0−odd)]_(8×42) =[d ₀]_(8×32) [SLI] _(32×32) [u] _(32×42)  (108)

and the desired field predictions are given by the following equation:$\begin{matrix}{{\left\lbrack {PL}_{0 - {odd}} \right\rbrack_{8 \times 42}\left\lbrack {xtb}^{\prime} \right\rbrack}_{42 \times 1} = \begin{bmatrix}\left\lbrack {\hat{x}{topi}^{\prime}} \right\rbrack_{4 \times 1} \\\left\lbrack {\hat{x}{boti}^{\prime}} \right\rbrack_{4 \times 1}\end{bmatrix}} & (109)\end{matrix}$

vmvtype—1

The down sampler for each field prediction is given by the following twoequations: $\begin{matrix}{\left\lbrack d_{1 - {top}} \right\rbrack_{4 \times 16} = \begin{bmatrix}0 & \lbrack f\rbrack_{8} & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & \lbrack f\rbrack_{8} & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 0 & 0 & \lbrack f\rbrack_{8} & 0 & 0 & 0 \\0 & 0 & 0 & 0 & 0 & 0 & 0 & \lbrack f\rbrack_{8} & 0\end{bmatrix}_{4 \times 16}} & (110) \\{\left\lbrack d_{1 - {bot}} \right\rbrack_{4 \times 16} = \begin{bmatrix}\lbrack f\rbrack_{8} & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & \lbrack f\rbrack_{8} & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 0 & \lbrack f\rbrack_{8} & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 0 & 0 & 0 & \lbrack f\rbrack_{8} & 0 & 0\end{bmatrix}_{4 \times 16}} & (111)\end{matrix}$

which may be combined into a single operator as given by the followingequation: $\begin{matrix}{\left\lbrack d_{1} \right\rbrack_{8 \times 32} = \begin{bmatrix}\left\lbrack d_{1 - {top}} \right\rbrack_{4 \times 16} & 0 \\0 & \left\lbrack d_{1 - {bot}} \right\rbrack_{4 \times 16}\end{bmatrix}} & (112)\end{matrix}$

Using this combined down sampling filter [d₁], combined filters[PL_(1−ev)]_(8×42) and [PL_(1−odd)]_(8×42) may be derived in a mannersimilar to the derivation of [PL_(0−ev)] and [PL_(0−0dd)] (shown inequations (101) and (108)), and may be used in accordance with thefollowing equation in order to produce the desired field prediction forthe vmv even valued case: $\begin{matrix}{{\left\lbrack {PL}_{1 - {ev}} \right\rbrack_{8 \times 42}\left\lbrack {xtb}^{\prime} \right\rbrack}_{42 \times 1} = \begin{bmatrix}\left\lbrack {\hat{x}{top}^{\prime}} \right\rbrack_{4 \times 1} \\\left\lbrack {\hat{x}{bot}^{\prime}} \right\rbrack_{4 \times 1}\end{bmatrix}} & (113)\end{matrix}$

and for the vmv odd valued case: $\begin{matrix}{{\left\lbrack {PL}_{1 - {odd}} \right\rbrack_{8 \times 42}\left\lbrack {xtb}^{\prime} \right\rbrack}_{42 \times 1} = {\begin{bmatrix}\left\lbrack {\hat{x}{topi}^{\prime}} \right\rbrack_{4 \times 1} \\\left\lbrack {\hat{x}{boti}^{\prime}} \right\rbrack_{4 \times 1}\end{bmatrix}.}} & (114)\end{matrix}$

vmvtype=2

The down sampling filter for each field prediction is given by thefollowing equation: $\begin{matrix}{\left\lbrack d_{2 - {top}} \right\rbrack_{4 \times 16} = {\left\lbrack d_{2 - {bot}} \right\rbrack_{4 \times 16} = \begin{bmatrix}0 & \lbrack f\rbrack_{8} & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & \lbrack f\rbrack_{8} & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 0 & 0 & \lbrack f\rbrack_{8} & 0 & 0 & 0 \\0 & 0 & 0 & 0 & 0 & 0 & 0 & \lbrack f\rbrack_{8} & 0\end{bmatrix}_{4 \times 16}}} & (115)\end{matrix}$

which can be combined according to the following equation:$\begin{matrix}{\left\lbrack d_{2} \right\rbrack_{8 \times 32} = \begin{bmatrix}\left\lbrack d_{2 - {top}} \right\rbrack_{4 \times 16} & 0 \\0 & \left\lbrack d_{2 - {bot}} \right\rbrack_{4 \times 16}\end{bmatrix}} & (116)\end{matrix}$

Using the down sampling filter of equation (116) in the same manner asthe derivations for vmvtype=0 and 1, the combined filters[PL_(2−ev)]_(8×42) and [PL_(2−odd)]_(8×42) can be derived and can beused according to the following equations in order to produce thedesired field prediction for the vmv even valued case: $\begin{matrix}{{\left\lbrack {PL}_{2 - {ev}} \right\rbrack_{8 \times 42}\left\lbrack {xtb}^{\prime} \right\rbrack}_{42 \times 1} = \begin{bmatrix}\left\lbrack {\hat{x}{top}^{\prime}} \right\rbrack_{4 \times 1} \\\left\lbrack {\hat{x}{bot}^{\prime}} \right\rbrack_{4 \times 1}\end{bmatrix}} & (117)\end{matrix}$

and for the vmv odd valued case: $\begin{matrix}{{\left\lbrack {PL}_{2 - {odd}} \right\rbrack_{8 \times 42}\left\lbrack {xtb}^{\prime} \right\rbrack}_{42 \times 1} = {\begin{bmatrix}\left\lbrack {\hat{x}{topi}^{\prime}} \right\rbrack_{4 \times 1} \\\left\lbrack {\hat{x}{boti}^{\prime}} \right\rbrack_{4 \times 1}\end{bmatrix}.}} & (118)\end{matrix}$

vmvtype=3

The down sampler for each field prediction is given by the following twoequations: $\begin{matrix}{\left\lbrack d_{3 - {top}} \right\rbrack_{4 \times 16} = \begin{bmatrix}0 & 0 & \lbrack f\rbrack_{8} & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 0 & \lbrack f\rbrack_{8} & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 0 & 0 & 0 & \lbrack f\rbrack_{8} & 0 & 0 \\0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & \lbrack f\rbrack_{8}\end{bmatrix}_{4 \times 16}} & (119) \\{\left\lbrack d_{3 - {bot}} \right\rbrack_{4 \times 16} = \begin{bmatrix}0 & \lbrack f\rbrack_{8} & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & \lbrack f\rbrack_{8} & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 0 & 0 & \lbrack f\rbrack_{8} & 0 & 0 & 0 \\0 & 0 & 0 & 0 & 0 & 0 & 0 & \lbrack f\rbrack_{8} & 0\end{bmatrix}_{4 \times 16}} & (120)\end{matrix}$

These two down sampling filters can be combined according to thefollowing equation: $\begin{matrix}{\left\lbrack d_{3} \right\rbrack_{8 \times 32} = \begin{bmatrix}\left\lbrack d_{3 - {top}} \right\rbrack_{4 \times 16} & 0 \\0 & \left\lbrack d_{3 - {bot}} \right\rbrack_{4 \times 16}\end{bmatrix}} & (121)\end{matrix}$

Using the down sampling filter of equation (121) in the same manner asthe derivations for vmvtype=0, 1, and 2, the combined filters[PL_(3−ev)]_(8×42) and [PL_(3−odd)]_(8×42) can be derived and can beused according to the following equations in order to produce thedesired field prediction for the vmv even valued case: $\begin{matrix}{{\left\lbrack {PL}_{3 - {ev}} \right\rbrack_{8 \times 42}\left\lbrack {xtb}^{\prime} \right\rbrack}_{42 \times 1} = \begin{bmatrix}\left\lbrack {\hat{x}{top}^{\prime}} \right\rbrack_{4 \times 1} \\\left\lbrack {\hat{x}{bot}^{\prime}} \right\rbrack_{4 \times 1}\end{bmatrix}} & (122)\end{matrix}$

and for the vmv odd valued case: $\begin{matrix}{{\left\lbrack {PL}_{3 - {odd}} \right\rbrack_{8 \times 42}\left\lbrack {xtb}^{\prime} \right\rbrack}_{42 \times 1} = {\begin{bmatrix}\left\lbrack {\hat{x}{topi}^{\prime}} \right\rbrack_{4 \times 1} \\\left\lbrack {\hat{x}{boti}^{\prime}} \right\rbrack_{4 \times 1}\end{bmatrix}.}} & (123)\end{matrix}$

IIC.—Summary of Derived Vertical Prediction Filter Operators

IIC1.—Field Prediction for Field Pictures

The following operators from equations (62), (66), (70), and (72) areused for field prediction of field pictures and are 8×25 matrices:

 [PL _(0−ev) ], [PL _(0−odd) ], [PL _(1−ev) ], [PL _(1−odd)]

IIC2.—Field Prediction for Frame Pictures

The following operators from equations (78), (83), (87), and (91) areused for field prediction of frame pictures and are 4×21 matrices, withtwo predictions required per macroblock:

[PL _(0−ev) ], [PL _(0−odd) ], [PL _(1−ev) ], [PL _(1−odd)]

IIC3.—Frame Prediction for Frame Pictures

The following operators from equations (101), (108), and similarequations are used for frame prediction of frame field pictures and are8×42 matrices:

[PL _(0−ev) ], [PL _(0−odd)]

[PL _(1−ev) ], [PL _(1−odd)]

[PL _(2−ev) ], [PL _(2−odd)]

[PL _(3−ev) ], [PL _(3−odd)]

IID.—Motion Compensator Block Diagram (FIG. 5, Module 104)

The motion compensator 104 is shown in more detail in FIG. 10. A motionvector translator 150 translates the received full resolution motionvector into a low resolution motion vector as previously described. Thislow resolution motion vector is then converted into reference picturememory addresses for supply to the memory 106. Vertically, thistranslation and conversion depends on the prediction type (frame orfield) as described in the preceding discussion. The reference pixelsread from the memory 106 by the addresses from the motion vectortranslator 150 are supplied to a horizontal prediction filter 152.Horizontal prediction filtering occurs in the horizontal predictionfilter 152 and is the same as previously described in connection withthe minimum drift prediction for a four point IDCT.

The results of the horizontal prediction filtering are supplied to avertical prediction filter 154. Vertical prediction filtering (upsampling and down sampling) is performed by the vertical predictionfilter 154 depending upon the prediction type and vmv value (odd oreven) as described above. Thus, depending on prediction type and vmvvalue (odd or even), the correct vertical filter operator [PL] is readfrom a memory 156 and is applied to each column of pixels as they arereceived from the horizontal prediction filter 152. The horizontally andvertically filtered prediction is supplied by the vertical predictionfilter 154 to an adder 108 (FIG. 5) to be added to the incoming residualmacroblock from the filtering and down sampling module 102.

The spatial effect of the vertical down sampling method implemented bythe filtering and down sampling module 102 of FIG. 5 must be considered.FIG. 11 shows the effective spatial relationship between the originalfull resolution pixels of an interlaced picture (pixels A for the topfield and pixels B for the bottom field as shown by the first column ofFIG. 11) and those of the corresponding down sampled interlaced picture(pixels “a” for the top field and pixels “b” for bottom field as shownby the second column of FIG. 11). It is noted that the derived “a”pixels fall half way between every other pair of A pixels, and thederived “b” pixels fall half way between every other pair of B pixels.It is also noted that, in the original full resolution picture, the Bpixels fall half way between the A pixels. However, it can be seen that,in the resulting down sampled picture, the “b” pixels do not fall halfway between the “a” pixels.

Therefore, an interpolator 110 of FIG. 5 contains vertical filters toeffectively adjust the vertical positions of the “a” and “b” fields sothat the “b” pixels fall midway between the “a” pixels as shown in FIG.12. One exemplary vertical filter consists of a simple linearinterpolator that shifts the “a” field up by ⅛ sample position and the“b” field down by ⅛ sample position. Filters more complicated thanlinear interpolators may also be used.

It is also possible to eliminate the need for the interpolator 110 ofFIG. 5 by modifying the filtering and down sampling module 102 of FIG. 5in order to achieve the above described ⅛ sample shifts for each of thefields. In this case, two different filters would be needed asreplacements for f₈ of equation (1) , one for the A or top field and onefor the B or bottom field. A down sampling filter for the A field filtergiven by the following equation would be derived similarly to that ofequation (1) but would include an ⅛ sample shift:

[f _(A)]=[−0.0104 −0.0057 0.1615 0.4130 0.3645 0.0973 −0.0156−0.0046]  (124)

Similarly, a down sampling filter for the B field given by the followingequation would include an ⅛ sample shift in the opposite direction:

[f _(B)]=[−0.0046 −0.0156 0.0973 0.3645 0.4130 0.1615 −0.0057−0.0104]  (125)

Respective matching prediction filters would have to be derived for eachof the fields in a manner exactly as previously described.

IIE.—Drift Reduction for The Case of ½ Pixel Resolution Motion Vectors

IIE1.—½ Pixel Interpolation in One Direction

As previously mentioned, motion vectors for MPEG-2 encoded video have ½pixel resolution. This ½ pixel resolution means that, when a vertical orhorizontal motion vector is odd valued, ½ pixel interpolation isrequired when forming a prediction from a full resolution reference oran up sampled reduced resolution reference, as is the case for the downconverting decoder 100 of FIG. 5. The MPEG-2 standard specifies that the½ pixel interpolation for the case of either an odd valued horizontalmotion vector or an odd valued vertical motion vector be calculatedaccording to the following equation: $\begin{matrix}\left\lfloor {\frac{x + y}{2} + 0.5} \right\rfloor & (126)\end{matrix}$

where x and y are integers (assumed to be 8 bits) representing thevalues of full resolution reference pixels.

This equation shows how to handle the least significant bit (LSB) of thecalculated interpolated pixel. If x+y=S, and if S is even, the aboveoperation is exactly equivalent to S/2. But if S is odd, then S/2=I.5,where I is an integer, and equation (126) will effectively cause a roundup to I+1. This increase from I.5 to I+1 is equivalent to an increase of½ LSB. Assuming that on average, half the time S is even and half thetime S is odd, equation (126) requires an average increase of ¼ LSB inthe calculated interpolated prediction pixel. A decoder that linearlycombines the calculation of S/2 with other filtering operations (likethe single step matrix operator of equation (66) executed by thevertical prediction filter 154 of FIG. 10) and that then rounds theresult will effectively retain the fractional part of S (S=I.5 50% ofthe time instead of the desired I+1) until final rounding. Such adecoder, therefore, exhibits a DC prediction drift towards a darkerpicture until the next I frame.

Therefore, to more closely meet the requirements of equation (126) foroperations such as that of equation (65) when the motion vector is oddvalued, the prediction output from the vertical prediction filter 154 ofFIG. 10 should be modified by adding 0.25 (equivalent to ¼ LSB) to itsoutput and then rounding the prediction result to an 8 bit integerbefore it is provided to the adder 108 of FIG. 5.

IIE2.—½ Pixel Interpolation in Both Directions

When both the horizontal and the vertical motion vectors are odd valued,the standard specifies a four way interpolation given by the followingequation: $\begin{matrix}\left\lfloor {\frac{a + b + c + d}{4} + 0.5} \right\rfloor & (127)\end{matrix}$

Equation (127) shows how to handle the LSB of the calculatedinterpolated pixel. If a+b+c+d=S, and if S is evenly divisible by four(case 1), the above operation is exactly equivalent to S/4. Otherwise,there are three other cases to consider (% denotes the modulooperation). In case 2, (S/4)%4=1, so that S/4=I.25. Equation (127) willreduce S/4 by 0.25 (−¼ LSB) . In case 3, (S/4)%4=2, so that S/4=I.5.Equation (127) will increase S/4 by 0.5 (+½ LSB). In case 4, (S/4)%4=3,so that S/4=I.75. Equation (127) will increase S/4 by 0.25 (+¼ LSB).

Assuming all four cases are equally probable, then on average equation(127) requires an increase of (0−0.25+0.5+0.25)/4=⅛ LSB in thecalculated interpolated prediction pixel. Therefore, for the case ofboth horizontal and vertical odd valued motion vectors, the predictionoutput of FIG. 10 should be modified by adding 0.125 (equivalent to ⅛LSB) to the output of the vertical prediction filter 154 and thenrounding the prediction result to an eight bit integer before it isprovided to the adder 108 of FIG. 5.

III.—Extension to Other Down Sampling Ratios

It should be noted that, in the preceding section, both the horizontaland vertical down sampling have been effected by a factor of two.However, the method described is not so restricted, and may be extendedto other down sampling ratios in a straightforward manner. For example,horizontal down sampling by utilizing a three point IDCT (instead of theabove mentioned four point IDCT) results in horizontal down sampling bya factor of 8/3 and is well known in the art. The novel vertical methodsdescribed herein can be extended to down sampling by a factor, forexample, of 8/5. A vertical filter f, similar to that of equation (1)may be used (perhaps one with somewhat less attenuation at higherfrequencies). The down sampling process can be decomposed into an upsample by five filter, then down sampled by eight. (Rational valued downsampling is well known). A minimum drift prediction filter may bederived from the down sampling filter f as previously described.

Certain modifications of the present invention have been discussedabove. Other modifications will occur to those practicing in the art ofthe present invention. For example, according to the description above,fourteen rows are used for performing a vertical IDCT during downconversion. However, any number of rows may instead be used.

Accordingly, the description of the present invention is to be construedas illustrative only and is for the purpose of teaching those skilled inthe art the best mode of carrying out the invention. The details may bevaried substantially without departing from the spirit of the invention,and the exclusive use of all modifications which are within the scope ofthe appended claims is reserved.

What is claimed is:
 1. A method of reducing drift in connection with anapplication of ½ pixel resolution motion vectors during predictionfiltering comprising the following steps: a) down converting received Pand B frame and field DCT coded blocks; b) selecting reference pixelsbased upon a received motion vector; c) performing prediction filteringon the selected reference pixels to produce a prediction output; d) ifthe motion vector is a ½ pixel resolution motion vector, adding a driftcompensation value to the prediction output; and, e) adding theprediction output to the down converted P and B frame and field DCTcoded blocks in order to form lower resolution reconstructed pixelblocks.
 2. The method of claim 1 wherein the motion vector includes botha vertical motion vector and a horizontal motion vector, and wherein thedrift compensation value is substantially 0.25 if one of the verticaland horizontal motion vectors is odd valued.
 3. The method of claim 1wherein the motion vector includes both a vertical motion vector and ahorizontal motion vector, and wherein the drift compensation value issubstantially 0.125 if both of the vertical and horizontal motionvectors are odd valued.
 4. The method of claim 1 wherein the motionvector includes both a vertical motion vector and a horizontal motionvector, and wherein the drift compensation value is substantially 0.25if one but not both of the vertical and horizontal motion vectors is oddvalued.
 5. The method of claim 1 wherein step d) comprises the step ofadding no drift compensation to the prediction output if the motionvector is not a ½ pixel resolution motion vector.
 6. The method of claim1 wherein step a) comprises the following steps: a1) converting thereceived P and B frame DCT coded blocks to converted P and B field DCTcoded blocks; and, a2) down converting the received P and B field DCTcoded blocks and the converted P and B field DCT coded blocks.
 7. Themethod of claim 6 wherein the motion vector includes both a verticalmotion vector and a horizontal motion vector, and wherein the driftcompensation value is substantially 0.25 if one of the vertical andhorizontal motion vectors is odd valued.
 8. The method of claim 6wherein the motion vector includes both a vertical motion vector and ahorizontal motion vector, and wherein the drift compensation value issubstantially 0.125 if both of the vertical and horizontal motionvectors are odd valued.
 9. The method of claim 6 wherein the motionvector includes both a vertical motion vector and a horizontal motionvector, and wherein the drift compensation value is substantially 0.25if one but not both of the vertical and horizontal motion vectors is oddvalued.
 10. The method of claim 6 wherein each of the received P and Bframe and field DCT coded blocks contains N×N values, wherein step a2)comprises the step of performing a horizontal M point IDCT, a vertical Npoint IDCT, vertical spatial filtering, and down sampling on thereceived P and B field DCT coded blocks and the converted P and B fieldDCT coded blocks, wherein at least the vertical spatial filtering anddown sampling encompasses more than N points, and wherein N>M.
 11. Themethod of claim 10 wherein the motion vector includes both a verticalmotion vector and a horizontal motion vector, and wherein the driftcompensation value is substantially 0.25 if one of the vertical andhorizontal motion vectors is odd valued.
 12. The method of claim 10wherein the motion vector includes both a vertical motion vector and ahorizontal motion vector, and wherein the drift compensation value issubstantially 0.125 if both of the vertical and horizontal motionvectors are odd valued.
 13. The method of claim 10 wherein the motionvector includes both a vertical motion vector and a horizontal motionvector, and wherein the drift compensation value is substantially 0.25if one but not both of the vertical and horizontal motion vectors is oddvalued.
 14. The method of claim 10 wherein N=8>M.
 15. The method ofclaim 14 wherein M=4.
 16. The method of claim 10 wherein the verticalspatial filtering and down sampling are combined as a single operator.17. The method of claim 10 wherein the vertical IDCT, spatial filtering,and down sampling are combined as a single operator.
 18. The method ofclaim 10 wherein step al) comprises the step of applying an unshufflingoperator to the received frame DCT coded blocks in order to convert thereceived frame DCT coded blocks to corresponding converted field DCTcoded blocks.
 19. A method of reconstructing pixel blocks from P and BDCT coded blocks comprising the following steps: a) selecting referencepixels from a memory in response to a motion vector; b) filtering theselected reference pixels to produce a filtered output; c) if the motionvector has ½ pixel resolution, adding a drift compensation value to thefiltered output; and, d) adding the filtered output to the P and B DCTcoded blocks in order to form the reconstructed pixel blocks.
 20. Themethod of claim 19 wherein the motion vector includes both a verticalmotion vector and a horizontal motion vector, and wherein the driftcompensation value is substantially 0.25 if one of the vertical andhorizontal motion vectors is odd valued.
 21. The method of claim 19wherein the motion vector includes both a vertical motion vector and ahorizontal motion vector, and wherein the drift compensation value issubstantially 0.125 if both of the vertical and horizontal motionvectors are odd valued.
 22. The method of claim 19 wherein the motionvector includes both a vertical motion vector and a horizontal motionvector, and wherein the drift compensation value is substantially 0.25if one but not both of the vertical and horizontal motion vectors is oddvalued.
 23. The method of claim 19 wherein step c) comprises the step ofadding no drift compensation to the prediction output if the motionvector is not a ½ pixel resolution motion vector.
 24. The method ofclaim 19 wherein the P and B DCT coded blocks are received P and B frameand field DCT coded blocks, and wherein step a) comprises the followingsteps: a1) converting the received P and B frame DCT coded blocks toconverted P and B field DCT coded blocks; a2) down converting thereceived P and B field DCT coded blocks and the converted P and B fieldDCT coded blocks; and, a3) selecting reference pixels from a memory inresponse to a motion vector.
 25. The method of claim 24 wherein themotion vector includes both a vertical motion vector and a horizontalmotion vector, and wherein the drift compensation value is substantially0.25 if one of the vertical and horizontal motion vectors is odd valued.26. The method of claim 24 wherein the motion vector includes both avertical motion vector and a horizontal motion vector, and wherein thedrift compensation value is substantially 0.125 if both of the verticaland horizontal motion vectors are odd valued.
 27. The method of claim 24wherein the motion vector includes both a vertical motion vector and ahorizontal motion vector, and wherein the drift compensation value issubstantially 0.25 if one but not both of the vertical and horizontalmotion vectors is odd valued.
 28. The method of claim 24 wherein each ofthe received P and B frame and field DCT coded blocks contains N×Nvalues, wherein step a2) comprises the step of performing a horizontal Mpoint IDCT, a vertical N point IDCT, vertical spatial filtering, anddown sampling on the received P and B field DCT coded blocks and theconverted P and B field DCT coded blocks, wherein at least the verticalspatial filtering and down sampling encompasses more than:N points, andwherein N>M.
 29. The method of claim 28 wherein the motion vectorincludes both a vertical motion vector and a horizontal motion vector,and wherein the drift compensation value is substantially 0.25 if one ofthe vertical and horizontal motion vectors is odd valued.
 30. The methodof claim 28 wherein the motion vector includes both a vertical motionvector and a horizontal motion vector, and wherein the driftcompensation value is substantially 0.125 if both of the vertical andhorizontal motion vectors are odd valued.
 31. The method of claim 28wherein the motion vector includes both a vertical motion vector and ahorizontal motion vector, and wherein the drift compensation value issubstantially 0.25 if one but not both of the vertical and horizontalmotion vectors is odd valued.
 32. The method of claim 28 wherein N=8>M.33. The method of claim 32 wherein M=4.
 34. The method of claim 28wherein the vertical spatial filtering and down sampling are combined asa single operator.
 35. The method of claim 28 wherein the vertical IDCT,spatial filtering, and down sampling are combined as a single operator.36. The method of claim 28 wherein step a1) comprises the step ofapplying an unshuffling operator to the received frame DCT coded blocksin order to convert the received frame DCT coded blocks to correspondingconverted field DCT coded blocks.
 37. A method of reconstructing pixelblocks from P and B DCT coded blocks comprising the following steps: a)selecting reference pixels from a memory in response to a motion vector;b) if the motion vector has ½ pixel resolution, adding driftcompensation to the selected reference pixels; and, c) adding theselected reference pixels to the P and B DCT coded blocks in order toform the reconstructed pixel blocks.
 38. The method of claim 37 whereinthe motion vector includes both a vertical motion vector and ahorizontal motion vector, and wherein the drift compensation is a valuesubstantially equal to 0.25 if one of the vertical and horizontal motionvectors is odd valued.
 39. The method of claim 37 wherein the motionvector includes both a vertical motion vector and a horizontal motionvector, and wherein the drift compensation is a value substantiallyequal to 0.125 if both of the vertical and horizontal motion vectors areodd valued.
 40. The method of claim 37 wherein the motion vectorincludes both a vertical motion vector and-a horizontal motion vector,and wherein the drift compensation is a value substantially equal to0.25 if one but not both of the vertical and horizontal motion vectorsis odd valued.
 41. The method of claim 37 wherein step b) comprises thestep of adding no drift compensation to the selected reference pixels ifthe motion vector is not a ½ pixel resolution motion vector.
 42. Themethod of claim 37 wherein the P and B DCT coded blocks are received Pand B frame and field DCT coded blocks, and wherein step a) comprisesthe following steps: a1) converting the received P and B frame DCT codedblocks to converted P and B field DCT coded blocks; a2) down convertingthe received P and B field DCT coded blocks and the converted P and Bfield DCT coded blocks; and, a3) selecting reference pixels from amemory in response to a motion vector.
 43. The method of claim 42wherein the motion vector includes both a vertical motion vector and ahorizontal motion vector, and wherein the drift compensation is a valuesubstantially equal to 0.25 if one of the vertical and horizontal motionvectors is odd valued.
 44. The method of claim 42 wherein the motionvector includes both a vertical motion vector and a horizontal motionvector, and wherein the drift compensation is a value substantiallyequal to 0.125 if both of the vertical and horizontal motion vectors areodd valued.
 45. The method of claim 42 wherein the motion vectorincludes both a vertical motion vector and a horizontal motion vector,and wherein the drift compensation is a value substantially equal to0.25 if one but not both of the vertical and horizontal motion vectorsis odd valued.
 46. The method of claim 42 wherein each of the received Pand B frame and field DCT coded blocks contains N×N values, wherein stepa2) comprises the step of performing a horizontal M point IDCT, avertical N point IDCT, vertical spatial filtering, and down sampling onthe received P and B field DCT coded blocks and the converted P and Bfield DCT coded blocks, wherein at least the vertical spatial filteringand down sampling encompasses more than N points, and wherein N>M. 47.The method of claim 46 wherein the motion vector includes both avertical motion vector and a horizontal motion vector, and wherein thedrift compensation is a value substantially equal to 0.25 if one of thevertical and horizontal motion vectors is odd valued.
 48. The method ofclaim 46 wherein the motion vector includes both a vertical motionvector and a horizontal motion vector, and wherein the driftcompensation is a value substantially equal to 0.125 if both of thevertical and horizontal motion vectors are odd valued.
 49. The method ofclaim 46 wherein the motion vector includes both a vertical motionvector and a horizontal motion vector, and wherein the driftcompensation is a value substantially equal to 0.25 if one but not bothof the vertical and horizontal motion vectors is odd valued.